2023-02-11 03:53:32,819 INFO [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.mob.compactions.TestMobCompactor timeout: 13 mins 2023-02-11 03:53:33,079 DEBUG [main] hbase.HBaseTestingUtility(342): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/3712ba8e-7346-8b4d-3d93-a55f0db4c73d 2023-02-11 03:53:33,094 INFO [Time-limited test] hbase.HBaseTestingUtility(1068): Starting up minicluster with option: StartMiniClusterOption{numMasters=1, masterClass=null, numRegionServers=1, rsPorts=, rsClass=null, numDataNodes=1, dataNodeHosts=null, numZkServers=1, createRootDir=false, createWALDir=false} 2023-02-11 03:53:33,095 INFO [Time-limited test] hbase.HBaseZKTestingUtility(82): Created new mini-cluster data directory: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/3712ba8e-7346-8b4d-3d93-a55f0db4c73d/cluster_d15eb7ef-e1c7-b786-c58e-de5925629f9c, deleteOnExit=true 2023-02-11 03:53:33,095 INFO [Time-limited test] hbase.HBaseTestingUtility(1082): STARTING DFS 2023-02-11 03:53:33,095 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting test.cache.data to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/3712ba8e-7346-8b4d-3d93-a55f0db4c73d/test.cache.data in system properties and HBase conf 2023-02-11 03:53:33,096 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting hadoop.tmp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/3712ba8e-7346-8b4d-3d93-a55f0db4c73d/hadoop.tmp.dir in system properties and HBase conf 2023-02-11 03:53:33,096 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting hadoop.log.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/3712ba8e-7346-8b4d-3d93-a55f0db4c73d/hadoop.log.dir in system properties and HBase conf 2023-02-11 03:53:33,097 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.local.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/3712ba8e-7346-8b4d-3d93-a55f0db4c73d/mapreduce.cluster.local.dir in system properties and HBase conf 2023-02-11 03:53:33,097 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.temp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/3712ba8e-7346-8b4d-3d93-a55f0db4c73d/mapreduce.cluster.temp.dir in system properties and HBase conf 2023-02-11 03:53:33,097 INFO [Time-limited test] hbase.HBaseTestingUtility(759): read short circuit is OFF 2023-02-11 03:53:33,214 WARN [Time-limited test] util.NativeCodeLoader(62): Unable to load native-hadoop library for your platform... using builtin-java classes where applicable 2023-02-11 03:53:33,626 DEBUG [Time-limited test] fs.HFileSystem(308): The file system is not a DistributedFileSystem. Skipping on block location reordering 2023-02-11 03:53:33,631 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.node-labels.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/3712ba8e-7346-8b4d-3d93-a55f0db4c73d/yarn.node-labels.fs-store.root-dir in system properties and HBase conf 2023-02-11 03:53:33,631 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.node-attribute.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/3712ba8e-7346-8b4d-3d93-a55f0db4c73d/yarn.node-attribute.fs-store.root-dir in system properties and HBase conf 2023-02-11 03:53:33,632 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.log-dirs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/3712ba8e-7346-8b4d-3d93-a55f0db4c73d/yarn.nodemanager.log-dirs in system properties and HBase conf 2023-02-11 03:53:33,632 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/3712ba8e-7346-8b4d-3d93-a55f0db4c73d/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-02-11 03:53:33,632 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.active-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/3712ba8e-7346-8b4d-3d93-a55f0db4c73d/yarn.timeline-service.entity-group-fs-store.active-dir in system properties and HBase conf 2023-02-11 03:53:33,633 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.done-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/3712ba8e-7346-8b4d-3d93-a55f0db4c73d/yarn.timeline-service.entity-group-fs-store.done-dir in system properties and HBase conf 2023-02-11 03:53:33,633 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/3712ba8e-7346-8b4d-3d93-a55f0db4c73d/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-02-11 03:53:33,633 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/3712ba8e-7346-8b4d-3d93-a55f0db4c73d/dfs.journalnode.edits.dir in system properties and HBase conf 2023-02-11 03:53:33,634 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.datanode.shared.file.descriptor.paths to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/3712ba8e-7346-8b4d-3d93-a55f0db4c73d/dfs.datanode.shared.file.descriptor.paths in system properties and HBase conf 2023-02-11 03:53:33,634 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting nfs.dump.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/3712ba8e-7346-8b4d-3d93-a55f0db4c73d/nfs.dump.dir in system properties and HBase conf 2023-02-11 03:53:33,634 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting java.io.tmpdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/3712ba8e-7346-8b4d-3d93-a55f0db4c73d/java.io.tmpdir in system properties and HBase conf 2023-02-11 03:53:33,635 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/3712ba8e-7346-8b4d-3d93-a55f0db4c73d/dfs.journalnode.edits.dir in system properties and HBase conf 2023-02-11 03:53:33,635 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.provided.aliasmap.inmemory.leveldb.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/3712ba8e-7346-8b4d-3d93-a55f0db4c73d/dfs.provided.aliasmap.inmemory.leveldb.dir in system properties and HBase conf 2023-02-11 03:53:33,635 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting fs.s3a.committer.staging.tmp.path to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/3712ba8e-7346-8b4d-3d93-a55f0db4c73d/fs.s3a.committer.staging.tmp.path in system properties and HBase conf Formatting using clusterid: testClusterID 2023-02-11 03:53:34,181 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-02-11 03:53:34,185 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-02-11 03:53:34,486 WARN [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-namenode.properties,hadoop-metrics2.properties 2023-02-11 03:53:34,665 INFO [Time-limited test] log.Slf4jLog(67): Logging to org.slf4j.impl.Reload4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog 2023-02-11 03:53:34,686 WARN [Time-limited test] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-02-11 03:53:34,737 INFO [Time-limited test] log.Slf4jLog(67): jetty-6.1.26 2023-02-11 03:53:34,782 INFO [Time-limited test] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/hdfs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/3712ba8e-7346-8b4d-3d93-a55f0db4c73d/java.io.tmpdir/Jetty_localhost_41515_hdfs____tier6u/webapp 2023-02-11 03:53:34,935 INFO [Time-limited test] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:41515 2023-02-11 03:53:34,977 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-02-11 03:53:34,977 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-02-11 03:53:35,453 WARN [Listener at localhost/45365] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-02-11 03:53:35,569 WARN [Listener at localhost/45365] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-02-11 03:53:35,587 WARN [Listener at localhost/45365] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-02-11 03:53:35,592 INFO [Listener at localhost/45365] log.Slf4jLog(67): jetty-6.1.26 2023-02-11 03:53:35,597 INFO [Listener at localhost/45365] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/3712ba8e-7346-8b4d-3d93-a55f0db4c73d/java.io.tmpdir/Jetty_localhost_43409_datanode____.r5w6uz/webapp 2023-02-11 03:53:35,691 INFO [Listener at localhost/45365] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:43409 2023-02-11 03:53:35,983 WARN [Listener at localhost/46205] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-02-11 03:53:36,402 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x1ccc45d515e2eb7f: Processing first storage report for DS-c30c3cb1-8ef7-45c5-a8b9-47f569ca4deb from datanode f8c64c2d-eaeb-46ff-bb7d-856787967a78 2023-02-11 03:53:36,403 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x1ccc45d515e2eb7f: from storage DS-c30c3cb1-8ef7-45c5-a8b9-47f569ca4deb node DatanodeRegistration(127.0.0.1:45197, datanodeUuid=f8c64c2d-eaeb-46ff-bb7d-856787967a78, infoPort=40297, infoSecurePort=0, ipcPort=46205, storageInfo=lv=-57;cid=testClusterID;nsid=231782251;c=1676087614267), blocks: 0, hasStaleStorage: true, processing time: 1 msecs, invalidatedBlocks: 0 2023-02-11 03:53:36,403 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x1ccc45d515e2eb7f: Processing first storage report for DS-b997c4b7-2894-4ae6-98ad-198d5b065622 from datanode f8c64c2d-eaeb-46ff-bb7d-856787967a78 2023-02-11 03:53:36,404 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x1ccc45d515e2eb7f: from storage DS-b997c4b7-2894-4ae6-98ad-198d5b065622 node DatanodeRegistration(127.0.0.1:45197, datanodeUuid=f8c64c2d-eaeb-46ff-bb7d-856787967a78, infoPort=40297, infoSecurePort=0, ipcPort=46205, storageInfo=lv=-57;cid=testClusterID;nsid=231782251;c=1676087614267), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-02-11 03:53:36,474 DEBUG [Listener at localhost/46205] hbase.HBaseTestingUtility(649): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/3712ba8e-7346-8b4d-3d93-a55f0db4c73d 2023-02-11 03:53:36,532 INFO [Listener at localhost/46205] zookeeper.MiniZooKeeperCluster(258): Started connectionTimeout=30000, dir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/3712ba8e-7346-8b4d-3d93-a55f0db4c73d/cluster_d15eb7ef-e1c7-b786-c58e-de5925629f9c/zookeeper_0, clientPort=53383, secureClientPort=-1, dataDir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/3712ba8e-7346-8b4d-3d93-a55f0db4c73d/cluster_d15eb7ef-e1c7-b786-c58e-de5925629f9c/zookeeper_0/version-2, dataDirSize=424 dataLogDir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/3712ba8e-7346-8b4d-3d93-a55f0db4c73d/cluster_d15eb7ef-e1c7-b786-c58e-de5925629f9c/zookeeper_0/version-2, dataLogSize=424 tickTime=2000, maxClientCnxns=300, minSessionTimeout=4000, maxSessionTimeout=40000, serverId=0 2023-02-11 03:53:36,545 INFO [Listener at localhost/46205] zookeeper.MiniZooKeeperCluster(283): Started MiniZooKeeperCluster and ran 'stat' on client port=53383 2023-02-11 03:53:36,552 INFO [Listener at localhost/46205] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-11 03:53:36,554 INFO [Listener at localhost/46205] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-11 03:53:37,190 INFO [Listener at localhost/46205] util.FSUtils(479): Created version file at hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac with version=8 2023-02-11 03:53:37,190 INFO [Listener at localhost/46205] hbase.HBaseTestingUtility(1406): Setting hbase.fs.tmp.dir to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/hbase-staging 2023-02-11 03:53:37,480 INFO [Listener at localhost/46205] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl 2023-02-11 03:53:37,908 INFO [Listener at localhost/46205] client.ConnectionUtils(127): master/jenkins-hbase4:0 server-side Connection retries=3 2023-02-11 03:53:37,939 INFO [Listener at localhost/46205] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-11 03:53:37,939 INFO [Listener at localhost/46205] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-02-11 03:53:37,939 INFO [Listener at localhost/46205] ipc.RWQueueRpcExecutor(105): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-02-11 03:53:37,940 INFO [Listener at localhost/46205] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-11 03:53:37,940 INFO [Listener at localhost/46205] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-02-11 03:53:38,078 INFO [Listener at localhost/46205] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.MasterService, hbase.pb.RegionServerStatusService, hbase.pb.LockService, hbase.pb.HbckService, hbase.pb.ClientMetaService, hbase.pb.ClientService, hbase.pb.AdminService 2023-02-11 03:53:38,156 DEBUG [Listener at localhost/46205] util.ClassSize(228): Using Unsafe to estimate memory layout 2023-02-11 03:53:38,247 INFO [Listener at localhost/46205] ipc.NettyRpcServer(120): Bind to /172.31.14.131:39921 2023-02-11 03:53:38,257 INFO [Listener at localhost/46205] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-11 03:53:38,258 INFO [Listener at localhost/46205] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-11 03:53:38,278 INFO [Listener at localhost/46205] zookeeper.RecoverableZooKeeper(93): Process identifier=master:39921 connecting to ZooKeeper ensemble=127.0.0.1:53383 2023-02-11 03:53:38,316 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:399210x0, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-02-11 03:53:38,318 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(623): master:39921-0x10041834dcb0000 connected 2023-02-11 03:53:38,344 DEBUG [Listener at localhost/46205] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-02-11 03:53:38,344 DEBUG [Listener at localhost/46205] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-11 03:53:38,347 DEBUG [Listener at localhost/46205] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-02-11 03:53:38,356 DEBUG [Listener at localhost/46205] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=39921 2023-02-11 03:53:38,356 DEBUG [Listener at localhost/46205] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=39921 2023-02-11 03:53:38,356 DEBUG [Listener at localhost/46205] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=39921 2023-02-11 03:53:38,356 DEBUG [Listener at localhost/46205] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=39921 2023-02-11 03:53:38,357 DEBUG [Listener at localhost/46205] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=39921 2023-02-11 03:53:38,362 INFO [Listener at localhost/46205] master.HMaster(439): hbase.rootdir=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac, hbase.cluster.distributed=false 2023-02-11 03:53:38,429 INFO [Listener at localhost/46205] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=3 2023-02-11 03:53:38,429 INFO [Listener at localhost/46205] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-11 03:53:38,429 INFO [Listener at localhost/46205] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-02-11 03:53:38,429 INFO [Listener at localhost/46205] ipc.RWQueueRpcExecutor(105): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-02-11 03:53:38,429 INFO [Listener at localhost/46205] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-11 03:53:38,430 INFO [Listener at localhost/46205] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-02-11 03:53:38,434 INFO [Listener at localhost/46205] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-02-11 03:53:38,437 INFO [Listener at localhost/46205] ipc.NettyRpcServer(120): Bind to /172.31.14.131:37011 2023-02-11 03:53:38,439 INFO [Listener at localhost/46205] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-02-11 03:53:38,445 DEBUG [Listener at localhost/46205] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-02-11 03:53:38,445 INFO [Listener at localhost/46205] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-11 03:53:38,448 INFO [Listener at localhost/46205] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-11 03:53:38,450 INFO [Listener at localhost/46205] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:37011 connecting to ZooKeeper ensemble=127.0.0.1:53383 2023-02-11 03:53:38,453 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:370110x0, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-02-11 03:53:38,454 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(623): regionserver:37011-0x10041834dcb0001 connected 2023-02-11 03:53:38,454 DEBUG [Listener at localhost/46205] zookeeper.ZKUtil(164): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-02-11 03:53:38,456 DEBUG [Listener at localhost/46205] zookeeper.ZKUtil(164): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-11 03:53:38,456 DEBUG [Listener at localhost/46205] zookeeper.ZKUtil(164): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-02-11 03:53:38,457 DEBUG [Listener at localhost/46205] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=37011 2023-02-11 03:53:38,457 DEBUG [Listener at localhost/46205] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=37011 2023-02-11 03:53:38,457 DEBUG [Listener at localhost/46205] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=37011 2023-02-11 03:53:38,458 DEBUG [Listener at localhost/46205] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=37011 2023-02-11 03:53:38,460 DEBUG [Listener at localhost/46205] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=37011 2023-02-11 03:53:38,462 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2158): Adding backup master ZNode /hbase/backup-masters/jenkins-hbase4.apache.org,39921,1676087617326 2023-02-11 03:53:38,470 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/backup-masters 2023-02-11 03:53:38,472 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on existing znode=/hbase/backup-masters/jenkins-hbase4.apache.org,39921,1676087617326 2023-02-11 03:53:38,490 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-02-11 03:53:38,490 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-02-11 03:53:38,490 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-11 03:53:38,491 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2023-02-11 03:53:38,493 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(224): Deleting ZNode for /hbase/backup-masters/jenkins-hbase4.apache.org,39921,1676087617326 from backup master directory 2023-02-11 03:53:38,493 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2023-02-11 03:53:38,497 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/backup-masters/jenkins-hbase4.apache.org,39921,1676087617326 2023-02-11 03:53:38,497 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/backup-masters 2023-02-11 03:53:38,498 WARN [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-02-11 03:53:38,498 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(234): Registered as active master=jenkins-hbase4.apache.org,39921,1676087617326 2023-02-11 03:53:38,500 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.ChunkCreator(497): Allocating data MemStoreChunkPool with chunk size 2 MB, max count 352, initial count 0 2023-02-11 03:53:38,502 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.ChunkCreator(497): Allocating index MemStoreChunkPool with chunk size 204.80 KB, max count 391, initial count 0 2023-02-11 03:53:38,580 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] util.FSUtils(628): Created cluster ID file at hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/hbase.id with ID: 1363d42f-8acc-4f9d-861b-33fa235a9a21 2023-02-11 03:53:38,619 INFO [master/jenkins-hbase4:0:becomeActiveMaster] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-11 03:53:38,633 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-11 03:53:38,669 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ReadOnlyZKClient(139): Connect 0x71412c82 to 127.0.0.1:53383 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-11 03:53:38,699 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@cd8ef1d, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-11 03:53:38,721 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(309): Create or load local region for table 'master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-02-11 03:53:38,723 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(132): Injected flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000 2023-02-11 03:53:38,739 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(264): ClientProtocol::create wrong number of arguments, should be hadoop 3.2 or below 2023-02-11 03:53:38,739 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(270): ClientProtocol::create wrong number of arguments, should be hadoop 2.x 2023-02-11 03:53:38,741 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(279): can not find SHOULD_REPLICATE flag, should be hadoop 2.x java.lang.IllegalArgumentException: No enum constant org.apache.hadoop.fs.CreateFlag.SHOULD_REPLICATE at java.lang.Enum.valueOf(Enum.java:238) at org.apache.hadoop.fs.CreateFlag.valueOf(CreateFlag.java:63) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.loadShouldReplicateFlag(FanOutOneBlockAsyncDFSOutputHelper.java:277) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.(FanOutOneBlockAsyncDFSOutputHelper.java:304) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.wal.AsyncFSWALProvider.load(AsyncFSWALProvider.java:139) at org.apache.hadoop.hbase.wal.WALFactory.getProviderClass(WALFactory.java:135) at org.apache.hadoop.hbase.wal.WALFactory.getProvider(WALFactory.java:175) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:202) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:182) at org.apache.hadoop.hbase.master.region.MasterRegion.create(MasterRegion.java:339) at org.apache.hadoop.hbase.master.region.MasterRegionFactory.create(MasterRegionFactory.java:104) at org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:849) at org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2178) at org.apache.hadoop.hbase.master.HMaster.lambda$run$0(HMaster.java:523) at java.lang.Thread.run(Thread.java:750) 2023-02-11 03:53:38,745 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(243): No decryptEncryptedDataEncryptionKey method in DFSClient, should be hadoop version with HDFS-12396 java.lang.NoSuchMethodException: org.apache.hadoop.hdfs.DFSClient.decryptEncryptedDataEncryptionKey(org.apache.hadoop.fs.FileEncryptionInfo) at java.lang.Class.getDeclaredMethod(Class.java:2130) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.createTransparentCryptoHelperWithoutHDFS12396(FanOutOneBlockAsyncDFSOutputSaslHelper.java:182) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.createTransparentCryptoHelper(FanOutOneBlockAsyncDFSOutputSaslHelper.java:241) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.(FanOutOneBlockAsyncDFSOutputSaslHelper.java:252) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.wal.AsyncFSWALProvider.load(AsyncFSWALProvider.java:140) at org.apache.hadoop.hbase.wal.WALFactory.getProviderClass(WALFactory.java:135) at org.apache.hadoop.hbase.wal.WALFactory.getProvider(WALFactory.java:175) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:202) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:182) at org.apache.hadoop.hbase.master.region.MasterRegion.create(MasterRegion.java:339) at org.apache.hadoop.hbase.master.region.MasterRegionFactory.create(MasterRegionFactory.java:104) at org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:849) at org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2178) at org.apache.hadoop.hbase.master.HMaster.lambda$run$0(HMaster.java:523) at java.lang.Thread.run(Thread.java:750) 2023-02-11 03:53:38,746 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-02-11 03:53:38,777 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7689): Creating {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, under table dir hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/MasterData/data/master/store-tmp 2023-02-11 03:53:39,203 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(865): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-11 03:53:39,204 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1603): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-02-11 03:53:39,204 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1625): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-11 03:53:39,204 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1646): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-11 03:53:39,204 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1713): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-02-11 03:53:39,204 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1723): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-11 03:53:39,204 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1837): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-11 03:53:39,205 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1557): Region close journal for 1595e783b53d99cd5eef43b6debb2682: Waiting for close lock at 1676087619203Disabling compacts and flushes for region at 1676087619204 (+1 ms)Disabling writes for close at 1676087619204Writing region close event to WAL at 1676087619204Closed at 1676087619204 2023-02-11 03:53:39,206 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(191): WALDir=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/MasterData/WALs/jenkins-hbase4.apache.org,39921,1676087617326 2023-02-11 03:53:39,226 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(464): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C39921%2C1676087617326, suffix=, logDir=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/MasterData/WALs/jenkins-hbase4.apache.org,39921,1676087617326, archiveDir=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/MasterData/oldWALs, maxLogs=10 2023-02-11 03:53:39,278 DEBUG [RS-EventLoopGroup-3-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:45197,DS-c30c3cb1-8ef7-45c5-a8b9-47f569ca4deb,DISK] 2023-02-11 03:53:39,286 DEBUG [RS-EventLoopGroup-3-2] asyncfs.ProtobufDecoder(123): Hadoop 3.2 and below use unshaded protobuf. java.lang.ClassNotFoundException: org.apache.hadoop.thirdparty.protobuf.MessageLite at java.net.URLClassLoader.findClass(URLClassLoader.java:387) at java.lang.ClassLoader.loadClass(ClassLoader.java:418) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352) at java.lang.ClassLoader.loadClass(ClassLoader.java:351) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.io.asyncfs.ProtobufDecoder.(ProtobufDecoder.java:118) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.processWriteBlockResponse(FanOutOneBlockAsyncDFSOutputHelper.java:340) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.access$100(FanOutOneBlockAsyncDFSOutputHelper.java:112) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$4.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:424) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:557) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.addListener(DefaultPromise.java:185) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.initialize(FanOutOneBlockAsyncDFSOutputHelper.java:418) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.access$300(FanOutOneBlockAsyncDFSOutputHelper.java:112) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$5.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:476) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$5.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:471) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setSuccess0(DefaultPromise.java:625) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.trySuccess(DefaultPromise.java:105) at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPromise.trySuccess(DefaultChannelPromise.java:84) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:653) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:691) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:489) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:397) at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.lang.Thread.run(Thread.java:750) 2023-02-11 03:53:39,352 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(758): New WAL /user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/MasterData/WALs/jenkins-hbase4.apache.org,39921,1676087617326/jenkins-hbase4.apache.org%2C39921%2C1676087617326.1676087619236 2023-02-11 03:53:39,353 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(839): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:45197,DS-c30c3cb1-8ef7-45c5-a8b9-47f569ca4deb,DISK]] 2023-02-11 03:53:39,353 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7850): Opening region: {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''} 2023-02-11 03:53:39,354 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(865): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-11 03:53:39,357 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7890): checking encryption for 1595e783b53d99cd5eef43b6debb2682 2023-02-11 03:53:39,358 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7893): checking classloading for 1595e783b53d99cd5eef43b6debb2682 2023-02-11 03:53:39,412 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family proc of region 1595e783b53d99cd5eef43b6debb2682 2023-02-11 03:53:39,419 DEBUG [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc 2023-02-11 03:53:39,442 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1595e783b53d99cd5eef43b6debb2682 columnFamilyName proc 2023-02-11 03:53:39,453 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(310): Store=1595e783b53d99cd5eef43b6debb2682/proc, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 03:53:39,459 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-02-11 03:53:39,460 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-02-11 03:53:39,476 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1054): writing seq id for 1595e783b53d99cd5eef43b6debb2682 2023-02-11 03:53:39,481 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-11 03:53:39,482 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1071): Opened 1595e783b53d99cd5eef43b6debb2682; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=67279771, jitterRate=0.0025467127561569214}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-11 03:53:39,482 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(964): Region open journal for 1595e783b53d99cd5eef43b6debb2682: Writing region info on filesystem at 1676087619383Initializing all the Stores at 1676087619385 (+2 ms)Instantiating store for column family {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676087619385Cleaning up temporary data from old regions at 1676087619467 (+82 ms)Cleaning up detritus from prior splits at 1676087619468 (+1 ms)Region opened successfully at 1676087619482 (+14 ms) 2023-02-11 03:53:39,483 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(122): Constructor flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000, compactMin=4 2023-02-11 03:53:39,502 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=5 2023-02-11 03:53:39,502 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(562): Starting 5 core workers (bigger of cpus/4 or 16) with max (burst) worker count=50 2023-02-11 03:53:39,504 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-11 03:53:39,506 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-11 03:53:39,537 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 30 msec 2023-02-11 03:53:39,537 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.RemoteProcedureDispatcher(95): Instantiated, coreThreads=3 (allowCoreThreadTimeOut=true), queueMaxSize=32, operationDelay=150 2023-02-11 03:53:39,561 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(253): hbase:meta replica znodes: [] 2023-02-11 03:53:39,566 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.RegionServerTracker(124): Starting RegionServerTracker; 0 have existing ServerCrashProcedures, 0 possibly 'live' servers, and 0 'splitting'. 2023-02-11 03:53:39,589 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.BaseLoadBalancer(1082): slop=0.001, systemTablesOnMaster=false 2023-02-11 03:53:39,592 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, ServerLocalityCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc. 2023-02-11 03:53:39,594 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/balancer 2023-02-11 03:53:39,599 INFO [master/jenkins-hbase4:0:becomeActiveMaster] normalizer.RegionNormalizerWorker(118): Normalizer rate limit set to unlimited 2023-02-11 03:53:39,603 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/normalizer 2023-02-11 03:53:39,606 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-11 03:53:39,607 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/switch/split 2023-02-11 03:53:39,608 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/switch/merge 2023-02-11 03:53:39,618 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/snapshot-cleanup 2023-02-11 03:53:39,623 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-02-11 03:53:39,623 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-02-11 03:53:39,623 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-11 03:53:39,624 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(739): Active/primary master=jenkins-hbase4.apache.org,39921,1676087617326, sessionid=0x10041834dcb0000, setting cluster-up flag (Was=false) 2023-02-11 03:53:39,637 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-11 03:53:39,643 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /hbase/flush-table-proc/acquired, /hbase/flush-table-proc/reached, /hbase/flush-table-proc/abort 2023-02-11 03:53:39,644 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,39921,1676087617326 2023-02-11 03:53:39,648 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-11 03:53:39,653 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /hbase/online-snapshot/acquired, /hbase/online-snapshot/reached, /hbase/online-snapshot/abort 2023-02-11 03:53:39,655 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,39921,1676087617326 2023-02-11 03:53:39,657 WARN [master/jenkins-hbase4:0:becomeActiveMaster] snapshot.SnapshotManager(302): Couldn't delete working snapshot directory: hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/.hbase-snapshot/.tmp 2023-02-11 03:53:39,664 INFO [RS:0;jenkins-hbase4:37011] regionserver.HRegionServer(952): ClusterId : 1363d42f-8acc-4f9d-861b-33fa235a9a21 2023-02-11 03:53:39,669 DEBUG [RS:0;jenkins-hbase4:37011] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-02-11 03:53:39,674 DEBUG [RS:0;jenkins-hbase4:37011] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-02-11 03:53:39,674 DEBUG [RS:0;jenkins-hbase4:37011] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-02-11 03:53:39,678 DEBUG [RS:0;jenkins-hbase4:37011] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-02-11 03:53:39,679 DEBUG [RS:0;jenkins-hbase4:37011] zookeeper.ReadOnlyZKClient(139): Connect 0x3c067254 to 127.0.0.1:53383 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-11 03:53:39,684 DEBUG [RS:0;jenkins-hbase4:37011] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@32c18ba4, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-11 03:53:39,684 DEBUG [RS:0;jenkins-hbase4:37011] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@36fe1924, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-02-11 03:53:39,708 DEBUG [RS:0;jenkins-hbase4:37011] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:0;jenkins-hbase4:37011 2023-02-11 03:53:39,711 INFO [RS:0;jenkins-hbase4:37011] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-02-11 03:53:39,711 INFO [RS:0;jenkins-hbase4:37011] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-02-11 03:53:39,711 DEBUG [RS:0;jenkins-hbase4:37011] regionserver.HRegionServer(1023): About to register with Master. 2023-02-11 03:53:39,714 INFO [RS:0;jenkins-hbase4:37011] regionserver.HRegionServer(2810): reportForDuty to master=jenkins-hbase4.apache.org,39921,1676087617326 with isa=jenkins-hbase4.apache.org/172.31.14.131:37011, startcode=1676087618428 2023-02-11 03:53:39,729 DEBUG [RS:0;jenkins-hbase4:37011] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-02-11 03:53:39,779 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:50709, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins.hfs.0 (auth:SIMPLE), service=RegionServerStatusService 2023-02-11 03:53:39,787 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] ipc.MetricsHBaseServer(134): Unknown exception type org.apache.hadoop.hbase.ipc.ServerNotRunningYetException: Server is not running yet at org.apache.hadoop.hbase.master.HMaster.checkServiceStarted(HMaster.java:2815) at org.apache.hadoop.hbase.master.MasterRpcServices.regionServerStartup(MasterRpcServices.java:577) at org.apache.hadoop.hbase.shaded.protobuf.generated.RegionServerStatusProtos$RegionServerStatusService$2.callBlockingMethod(RegionServerStatusProtos.java:15952) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) 2023-02-11 03:53:39,788 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT; InitMetaProcedure table=hbase:meta 2023-02-11 03:53:39,797 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_OPEN_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-02-11 03:53:39,797 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_CLOSE_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-02-11 03:53:39,798 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-02-11 03:53:39,798 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_META_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-02-11 03:53:39,798 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=M_LOG_REPLAY_OPS-master/jenkins-hbase4:0, corePoolSize=10, maxPoolSize=10 2023-02-11 03:53:39,798 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-11 03:53:39,798 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_MERGE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-02-11 03:53:39,798 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_TABLE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-11 03:53:39,800 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676087649800 2023-02-11 03:53:39,802 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): log_cleaner Cleaner pool size is 1 2023-02-11 03:53:39,805 DEBUG [PEWorker-2] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT, locked=true; InitMetaProcedure table=hbase:meta 2023-02-11 03:53:39,805 INFO [PEWorker-2] procedure.InitMetaProcedure(71): BOOTSTRAP: creating hbase:meta region 2023-02-11 03:53:39,811 INFO [PEWorker-2] util.FSTableDescriptors(128): Creating new hbase:meta table descriptor 'hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-02-11 03:53:39,814 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveLogCleaner 2023-02-11 03:53:39,814 DEBUG [RS:0;jenkins-hbase4:37011] regionserver.HRegionServer(2829): Master is not running yet 2023-02-11 03:53:39,814 WARN [RS:0;jenkins-hbase4:37011] regionserver.HRegionServer(1031): reportForDuty failed; sleeping 100 ms and then retrying. 2023-02-11 03:53:39,820 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.replication.master.ReplicationLogCleaner 2023-02-11 03:53:39,821 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreWALCleaner 2023-02-11 03:53:39,821 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveProcedureWALCleaner 2023-02-11 03:53:39,821 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.LogCleaner(148): Creating 1 old WALs cleaner threads 2023-02-11 03:53:39,822 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=LogsCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-02-11 03:53:39,823 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): hfile_cleaner Cleaner pool size is 2 2023-02-11 03:53:39,824 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreHFileCleaner 2023-02-11 03:53:39,825 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.HFileLinkCleaner 2023-02-11 03:53:39,827 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.snapshot.SnapshotHFileCleaner 2023-02-11 03:53:39,828 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveHFileCleaner 2023-02-11 03:53:39,829 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(242): Starting for large file=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1676087619829,5,FailOnTimeoutGroup] 2023-02-11 03:53:39,830 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(257): Starting for small files=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1676087619829,5,FailOnTimeoutGroup] 2023-02-11 03:53:39,830 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HFileCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-02-11 03:53:39,830 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1451): Reopening regions with very high storeFileRefCount is disabled. Provide threshold value > 0 for hbase.regions.recovery.store.file.ref.count to enable it. 2023-02-11 03:53:39,831 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=ReplicationBarrierCleaner, period=43200000, unit=MILLISECONDS is enabled. 2023-02-11 03:53:39,832 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=SnapshotCleaner, period=1800000, unit=MILLISECONDS is enabled. 2023-02-11 03:53:39,838 DEBUG [PEWorker-2] util.FSTableDescriptors(570): Wrote into hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-02-11 03:53:39,839 INFO [PEWorker-2] util.FSTableDescriptors(135): Updated hbase:meta table descriptor to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-02-11 03:53:39,839 INFO [PEWorker-2] regionserver.HRegion(7671): creating {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac 2023-02-11 03:53:39,849 DEBUG [PEWorker-2] regionserver.HRegion(865): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-11 03:53:39,851 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-02-11 03:53:39,854 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/hbase/meta/1588230740/info 2023-02-11 03:53:39,855 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-02-11 03:53:39,856 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 03:53:39,856 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-02-11 03:53:39,859 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/hbase/meta/1588230740/rep_barrier 2023-02-11 03:53:39,859 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-02-11 03:53:39,860 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 03:53:39,861 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-02-11 03:53:39,862 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/hbase/meta/1588230740/table 2023-02-11 03:53:39,863 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-02-11 03:53:39,864 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 03:53:39,865 DEBUG [PEWorker-2] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/hbase/meta/1588230740 2023-02-11 03:53:39,866 DEBUG [PEWorker-2] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/hbase/meta/1588230740 2023-02-11 03:53:39,869 DEBUG [PEWorker-2] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-02-11 03:53:39,871 DEBUG [PEWorker-2] regionserver.HRegion(1054): writing seq id for 1588230740 2023-02-11 03:53:39,875 DEBUG [PEWorker-2] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/hbase/meta/1588230740/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-11 03:53:39,875 INFO [PEWorker-2] regionserver.HRegion(1071): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=74128547, jitterRate=0.1046014279127121}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-02-11 03:53:39,876 DEBUG [PEWorker-2] regionserver.HRegion(964): Region open journal for 1588230740: Writing region info on filesystem at 1676087619849Initializing all the Stores at 1676087619850 (+1 ms)Instantiating store for column family {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} at 1676087619851 (+1 ms)Instantiating store for column family {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676087619851Instantiating store for column family {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} at 1676087619851Cleaning up temporary data from old regions at 1676087619868 (+17 ms)Cleaning up detritus from prior splits at 1676087619868Region opened successfully at 1676087619876 (+8 ms) 2023-02-11 03:53:39,876 DEBUG [PEWorker-2] regionserver.HRegion(1603): Closing 1588230740, disabling compactions & flushes 2023-02-11 03:53:39,876 INFO [PEWorker-2] regionserver.HRegion(1625): Closing region hbase:meta,,1.1588230740 2023-02-11 03:53:39,876 DEBUG [PEWorker-2] regionserver.HRegion(1646): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-02-11 03:53:39,876 DEBUG [PEWorker-2] regionserver.HRegion(1713): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-02-11 03:53:39,876 DEBUG [PEWorker-2] regionserver.HRegion(1723): Updates disabled for region hbase:meta,,1.1588230740 2023-02-11 03:53:39,877 INFO [PEWorker-2] regionserver.HRegion(1837): Closed hbase:meta,,1.1588230740 2023-02-11 03:53:39,877 DEBUG [PEWorker-2] regionserver.HRegion(1557): Region close journal for 1588230740: Waiting for close lock at 1676087619876Disabling compacts and flushes for region at 1676087619876Disabling writes for close at 1676087619876Writing region close event to WAL at 1676087619877 (+1 ms)Closed at 1676087619877 2023-02-11 03:53:39,882 DEBUG [PEWorker-2] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_ASSIGN_META, locked=true; InitMetaProcedure table=hbase:meta 2023-02-11 03:53:39,882 INFO [PEWorker-2] procedure.InitMetaProcedure(103): Going to assign meta 2023-02-11 03:53:39,890 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN}] 2023-02-11 03:53:39,901 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN 2023-02-11 03:53:39,903 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN; state=OFFLINE, location=null; forceNewPlan=false, retain=false 2023-02-11 03:53:39,915 INFO [RS:0;jenkins-hbase4:37011] regionserver.HRegionServer(2810): reportForDuty to master=jenkins-hbase4.apache.org,39921,1676087617326 with isa=jenkins-hbase4.apache.org/172.31.14.131:37011, startcode=1676087618428 2023-02-11 03:53:39,920 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:53:39,925 DEBUG [RS:0;jenkins-hbase4:37011] regionserver.HRegionServer(1596): Config from master: hbase.rootdir=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac 2023-02-11 03:53:39,925 DEBUG [RS:0;jenkins-hbase4:37011] regionserver.HRegionServer(1596): Config from master: fs.defaultFS=hdfs://localhost:45365 2023-02-11 03:53:39,925 DEBUG [RS:0;jenkins-hbase4:37011] regionserver.HRegionServer(1596): Config from master: hbase.master.info.port=-1 2023-02-11 03:53:39,930 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-11 03:53:39,931 DEBUG [RS:0;jenkins-hbase4:37011] zookeeper.ZKUtil(162): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:53:39,931 WARN [RS:0;jenkins-hbase4:37011] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-02-11 03:53:39,931 INFO [RS:0;jenkins-hbase4:37011] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-02-11 03:53:39,932 DEBUG [RS:0;jenkins-hbase4:37011] regionserver.HRegionServer(1947): logDir=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/WALs/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:53:39,932 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,37011,1676087618428] 2023-02-11 03:53:39,939 DEBUG [RS:0;jenkins-hbase4:37011] zookeeper.ZKUtil(162): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:53:39,949 DEBUG [RS:0;jenkins-hbase4:37011] regionserver.Replication(139): Replication stats-in-log period=300 seconds 2023-02-11 03:53:39,957 INFO [RS:0;jenkins-hbase4:37011] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-02-11 03:53:39,975 INFO [RS:0;jenkins-hbase4:37011] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-02-11 03:53:39,978 INFO [RS:0;jenkins-hbase4:37011] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-11 03:53:39,978 INFO [RS:0;jenkins-hbase4:37011] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-11 03:53:39,979 INFO [RS:0;jenkins-hbase4:37011] regionserver.HRegionServer$CompactionChecker(1838): CompactionChecker runs every PT1S 2023-02-11 03:53:39,985 INFO [RS:0;jenkins-hbase4:37011] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=100, unit=MILLISECONDS is enabled. 2023-02-11 03:53:39,985 DEBUG [RS:0;jenkins-hbase4:37011] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-11 03:53:39,986 DEBUG [RS:0;jenkins-hbase4:37011] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-11 03:53:39,986 DEBUG [RS:0;jenkins-hbase4:37011] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-11 03:53:39,986 DEBUG [RS:0;jenkins-hbase4:37011] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-11 03:53:39,986 DEBUG [RS:0;jenkins-hbase4:37011] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-11 03:53:39,986 DEBUG [RS:0;jenkins-hbase4:37011] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-02-11 03:53:39,986 DEBUG [RS:0;jenkins-hbase4:37011] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-11 03:53:39,986 DEBUG [RS:0;jenkins-hbase4:37011] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-11 03:53:39,986 DEBUG [RS:0;jenkins-hbase4:37011] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-11 03:53:39,987 DEBUG [RS:0;jenkins-hbase4:37011] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-11 03:53:39,987 INFO [RS:0;jenkins-hbase4:37011] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-02-11 03:53:39,987 INFO [RS:0;jenkins-hbase4:37011] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-02-11 03:53:39,988 INFO [RS:0;jenkins-hbase4:37011] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-02-11 03:53:40,002 INFO [RS:0;jenkins-hbase4:37011] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-02-11 03:53:40,004 INFO [RS:0;jenkins-hbase4:37011] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,37011,1676087618428-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-02-11 03:53:40,019 INFO [RS:0;jenkins-hbase4:37011] regionserver.Replication(203): jenkins-hbase4.apache.org,37011,1676087618428 started 2023-02-11 03:53:40,019 INFO [RS:0;jenkins-hbase4:37011] regionserver.HRegionServer(1638): Serving as jenkins-hbase4.apache.org,37011,1676087618428, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:37011, sessionid=0x10041834dcb0001 2023-02-11 03:53:40,020 DEBUG [RS:0;jenkins-hbase4:37011] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-02-11 03:53:40,020 DEBUG [RS:0;jenkins-hbase4:37011] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:53:40,020 DEBUG [RS:0;jenkins-hbase4:37011] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,37011,1676087618428' 2023-02-11 03:53:40,020 DEBUG [RS:0;jenkins-hbase4:37011] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:53:40,020 DEBUG [RS:0;jenkins-hbase4:37011] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:53:40,021 DEBUG [RS:0;jenkins-hbase4:37011] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-02-11 03:53:40,021 DEBUG [RS:0;jenkins-hbase4:37011] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-02-11 03:53:40,021 DEBUG [RS:0;jenkins-hbase4:37011] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:53:40,021 DEBUG [RS:0;jenkins-hbase4:37011] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,37011,1676087618428' 2023-02-11 03:53:40,021 DEBUG [RS:0;jenkins-hbase4:37011] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-02-11 03:53:40,022 DEBUG [RS:0;jenkins-hbase4:37011] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-02-11 03:53:40,022 DEBUG [RS:0;jenkins-hbase4:37011] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-02-11 03:53:40,022 INFO [RS:0;jenkins-hbase4:37011] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-02-11 03:53:40,022 INFO [RS:0;jenkins-hbase4:37011] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-02-11 03:53:40,055 DEBUG [jenkins-hbase4:39921] assignment.AssignmentManager(2178): Processing assignQueue; systemServersCount=1, allServersCount=1 2023-02-11 03:53:40,057 INFO [PEWorker-3] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,37011,1676087618428, state=OPENING 2023-02-11 03:53:40,064 DEBUG [PEWorker-3] zookeeper.MetaTableLocator(240): hbase:meta region location doesn't exist, create it 2023-02-11 03:53:40,067 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-11 03:53:40,067 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /hbase/meta-region-server: CHANGED 2023-02-11 03:53:40,070 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=3, ppid=2, state=RUNNABLE; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,37011,1676087618428}] 2023-02-11 03:53:40,133 INFO [RS:0;jenkins-hbase4:37011] wal.AbstractFSWAL(464): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C37011%2C1676087618428, suffix=, logDir=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/WALs/jenkins-hbase4.apache.org,37011,1676087618428, archiveDir=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/oldWALs, maxLogs=32 2023-02-11 03:53:40,152 DEBUG [RS-EventLoopGroup-3-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:45197,DS-c30c3cb1-8ef7-45c5-a8b9-47f569ca4deb,DISK] 2023-02-11 03:53:40,156 INFO [RS:0;jenkins-hbase4:37011] wal.AbstractFSWAL(758): New WAL /user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/WALs/jenkins-hbase4.apache.org,37011,1676087618428/jenkins-hbase4.apache.org%2C37011%2C1676087618428.1676087620136 2023-02-11 03:53:40,157 DEBUG [RS:0;jenkins-hbase4:37011] wal.AbstractFSWAL(839): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:45197,DS-c30c3cb1-8ef7-45c5-a8b9-47f569ca4deb,DISK]] 2023-02-11 03:53:40,253 DEBUG [RSProcedureDispatcher-pool-0] master.ServerManager(712): New admin connection to jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:53:40,256 DEBUG [RSProcedureDispatcher-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-02-11 03:53:40,259 INFO [RS-EventLoopGroup-3-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:55366, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-02-11 03:53:40,270 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open hbase:meta,,1.1588230740 2023-02-11 03:53:40,270 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-02-11 03:53:40,274 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(464): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C37011%2C1676087618428.meta, suffix=.meta, logDir=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/WALs/jenkins-hbase4.apache.org,37011,1676087618428, archiveDir=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/oldWALs, maxLogs=32 2023-02-11 03:53:40,293 DEBUG [RS-EventLoopGroup-3-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:45197,DS-c30c3cb1-8ef7-45c5-a8b9-47f569ca4deb,DISK] 2023-02-11 03:53:40,297 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(758): New WAL /user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/WALs/jenkins-hbase4.apache.org,37011,1676087618428/jenkins-hbase4.apache.org%2C37011%2C1676087618428.meta.1676087620276.meta 2023-02-11 03:53:40,297 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(839): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:45197,DS-c30c3cb1-8ef7-45c5-a8b9-47f569ca4deb,DISK]] 2023-02-11 03:53:40,298 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''} 2023-02-11 03:53:40,300 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(215): Loading coprocessor class org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint with path null and priority 536870911 2023-02-11 03:53:40,318 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(8546): Registered coprocessor service: region=hbase:meta,,1 service=MultiRowMutationService 2023-02-11 03:53:40,323 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.RegionCoprocessorHost(393): Loaded coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint from HTD of hbase:meta successfully. 2023-02-11 03:53:40,329 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table meta 1588230740 2023-02-11 03:53:40,329 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-11 03:53:40,329 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 1588230740 2023-02-11 03:53:40,329 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 1588230740 2023-02-11 03:53:40,332 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-02-11 03:53:40,333 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/hbase/meta/1588230740/info 2023-02-11 03:53:40,333 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/hbase/meta/1588230740/info 2023-02-11 03:53:40,334 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-02-11 03:53:40,335 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 03:53:40,335 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-02-11 03:53:40,336 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/hbase/meta/1588230740/rep_barrier 2023-02-11 03:53:40,336 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/hbase/meta/1588230740/rep_barrier 2023-02-11 03:53:40,337 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-02-11 03:53:40,338 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 03:53:40,338 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-02-11 03:53:40,339 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/hbase/meta/1588230740/table 2023-02-11 03:53:40,339 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/hbase/meta/1588230740/table 2023-02-11 03:53:40,340 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-02-11 03:53:40,341 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 03:53:40,343 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/hbase/meta/1588230740 2023-02-11 03:53:40,345 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/hbase/meta/1588230740 2023-02-11 03:53:40,349 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-02-11 03:53:40,351 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 1588230740 2023-02-11 03:53:40,353 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=70431100, jitterRate=0.04950517416000366}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-02-11 03:53:40,353 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 1588230740: Running coprocessor pre-open hook at 1676087620329Writing region info on filesystem at 1676087620330 (+1 ms)Initializing all the Stores at 1676087620331 (+1 ms)Instantiating store for column family {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} at 1676087620331Instantiating store for column family {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676087620332 (+1 ms)Instantiating store for column family {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} at 1676087620332Cleaning up temporary data from old regions at 1676087620347 (+15 ms)Cleaning up detritus from prior splits at 1676087620348 (+1 ms)Running coprocessor post-open hooks at 1676087620353 (+5 ms)Region opened successfully at 1676087620353 2023-02-11 03:53:40,362 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for hbase:meta,,1.1588230740, pid=3, masterSystemTime=1676087620246 2023-02-11 03:53:40,380 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for hbase:meta,,1.1588230740 2023-02-11 03:53:40,380 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened hbase:meta,,1.1588230740 2023-02-11 03:53:40,381 INFO [PEWorker-5] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,37011,1676087618428, state=OPEN 2023-02-11 03:53:40,384 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/meta-region-server 2023-02-11 03:53:40,384 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /hbase/meta-region-server: CHANGED 2023-02-11 03:53:40,388 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=3, resume processing ppid=2 2023-02-11 03:53:40,388 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=3, ppid=2, state=SUCCESS; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,37011,1676087618428 in 314 msec 2023-02-11 03:53:40,392 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=2, resume processing ppid=1 2023-02-11 03:53:40,393 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=2, ppid=1, state=SUCCESS; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN in 500 msec 2023-02-11 03:53:40,398 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta in 686 msec 2023-02-11 03:53:40,398 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(946): Master startup: status=Wait for region servers to report in, state=RUNNING, startTime=1676087618487, completionTime=-1 2023-02-11 03:53:40,398 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(821): Finished waiting on RegionServer count=1; waited=0ms, expected min=1 server(s), max=1 server(s), master is running 2023-02-11 03:53:40,398 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1519): Joining cluster... 2023-02-11 03:53:40,456 DEBUG [hconnection-0x2fbe5b7a-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-11 03:53:40,460 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:55372, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-11 03:53:40,477 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1531): Number of RegionServers=1 2023-02-11 03:53:40,477 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676087680477 2023-02-11 03:53:40,477 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676087740477 2023-02-11 03:53:40,477 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1538): Joined the cluster in 78 msec 2023-02-11 03:53:40,500 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,39921,1676087617326-ClusterStatusChore, period=60000, unit=MILLISECONDS is enabled. 2023-02-11 03:53:40,500 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,39921,1676087617326-BalancerChore, period=300000, unit=MILLISECONDS is enabled. 2023-02-11 03:53:40,500 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,39921,1676087617326-RegionNormalizerChore, period=300000, unit=MILLISECONDS is enabled. 2023-02-11 03:53:40,502 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=CatalogJanitor-jenkins-hbase4:39921, period=300000, unit=MILLISECONDS is enabled. 2023-02-11 03:53:40,502 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HbckChore-, period=3600000, unit=MILLISECONDS is enabled. 2023-02-11 03:53:40,508 DEBUG [master/jenkins-hbase4:0.Chore.1] janitor.CatalogJanitor(175): 2023-02-11 03:53:40,516 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.TableNamespaceManager(92): Namespace table not found. Creating... 2023-02-11 03:53:40,517 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2138): Client=null/null create 'hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-02-11 03:53:40,526 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=hbase:namespace 2023-02-11 03:53:40,528 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_PRE_OPERATION 2023-02-11 03:53:40,531 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-11 03:53:40,551 DEBUG [HFileArchiver-1] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/.tmp/data/hbase/namespace/ebab26be0da503a9bd5db0a862dc884b 2023-02-11 03:53:40,553 DEBUG [HFileArchiver-1] backup.HFileArchiver(153): Directory hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/.tmp/data/hbase/namespace/ebab26be0da503a9bd5db0a862dc884b empty. 2023-02-11 03:53:40,554 DEBUG [HFileArchiver-1] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/.tmp/data/hbase/namespace/ebab26be0da503a9bd5db0a862dc884b 2023-02-11 03:53:40,554 DEBUG [PEWorker-3] procedure.DeleteTableProcedure(328): Archived hbase:namespace regions 2023-02-11 03:53:40,977 DEBUG [PEWorker-3] util.FSTableDescriptors(570): Wrote into hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/.tmp/data/hbase/namespace/.tabledesc/.tableinfo.0000000001 2023-02-11 03:53:40,979 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(7671): creating {ENCODED => ebab26be0da503a9bd5db0a862dc884b, NAME => 'hbase:namespace,,1676087620517.ebab26be0da503a9bd5db0a862dc884b.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/.tmp 2023-02-11 03:53:41,391 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(865): Instantiated hbase:namespace,,1676087620517.ebab26be0da503a9bd5db0a862dc884b.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-11 03:53:41,391 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1603): Closing ebab26be0da503a9bd5db0a862dc884b, disabling compactions & flushes 2023-02-11 03:53:41,392 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1625): Closing region hbase:namespace,,1676087620517.ebab26be0da503a9bd5db0a862dc884b. 2023-02-11 03:53:41,392 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on hbase:namespace,,1676087620517.ebab26be0da503a9bd5db0a862dc884b. 2023-02-11 03:53:41,392 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1713): Acquired close lock on hbase:namespace,,1676087620517.ebab26be0da503a9bd5db0a862dc884b. after waiting 0 ms 2023-02-11 03:53:41,392 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1723): Updates disabled for region hbase:namespace,,1676087620517.ebab26be0da503a9bd5db0a862dc884b. 2023-02-11 03:53:41,392 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1837): Closed hbase:namespace,,1676087620517.ebab26be0da503a9bd5db0a862dc884b. 2023-02-11 03:53:41,392 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1557): Region close journal for ebab26be0da503a9bd5db0a862dc884b: Waiting for close lock at 1676087621391Disabling compacts and flushes for region at 1676087621391Disabling writes for close at 1676087621392 (+1 ms)Writing region close event to WAL at 1676087621392Closed at 1676087621392 2023-02-11 03:53:41,397 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ADD_TO_META 2023-02-11 03:53:41,411 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"hbase:namespace,,1676087620517.ebab26be0da503a9bd5db0a862dc884b.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1676087621399"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676087621399"}]},"ts":"1676087621399"} 2023-02-11 03:53:41,438 INFO [PEWorker-3] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-11 03:53:41,440 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-11 03:53:41,445 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676087621440"}]},"ts":"1676087621440"} 2023-02-11 03:53:41,449 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLING in hbase:meta 2023-02-11 03:53:41,458 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=ebab26be0da503a9bd5db0a862dc884b, ASSIGN}] 2023-02-11 03:53:41,461 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=ebab26be0da503a9bd5db0a862dc884b, ASSIGN 2023-02-11 03:53:41,462 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:namespace, region=ebab26be0da503a9bd5db0a862dc884b, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,37011,1676087618428; forceNewPlan=false, retain=false 2023-02-11 03:53:41,613 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=ebab26be0da503a9bd5db0a862dc884b, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:53:41,614 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"hbase:namespace,,1676087620517.ebab26be0da503a9bd5db0a862dc884b.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1676087621613"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676087621613"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676087621613"}]},"ts":"1676087621613"} 2023-02-11 03:53:41,618 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=6, ppid=5, state=RUNNABLE; OpenRegionProcedure ebab26be0da503a9bd5db0a862dc884b, server=jenkins-hbase4.apache.org,37011,1676087618428}] 2023-02-11 03:53:41,778 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open hbase:namespace,,1676087620517.ebab26be0da503a9bd5db0a862dc884b. 2023-02-11 03:53:41,779 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => ebab26be0da503a9bd5db0a862dc884b, NAME => 'hbase:namespace,,1676087620517.ebab26be0da503a9bd5db0a862dc884b.', STARTKEY => '', ENDKEY => ''} 2023-02-11 03:53:41,779 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table namespace ebab26be0da503a9bd5db0a862dc884b 2023-02-11 03:53:41,779 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated hbase:namespace,,1676087620517.ebab26be0da503a9bd5db0a862dc884b.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-11 03:53:41,780 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for ebab26be0da503a9bd5db0a862dc884b 2023-02-11 03:53:41,780 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for ebab26be0da503a9bd5db0a862dc884b 2023-02-11 03:53:41,782 INFO [StoreOpener-ebab26be0da503a9bd5db0a862dc884b-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region ebab26be0da503a9bd5db0a862dc884b 2023-02-11 03:53:41,784 DEBUG [StoreOpener-ebab26be0da503a9bd5db0a862dc884b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/hbase/namespace/ebab26be0da503a9bd5db0a862dc884b/info 2023-02-11 03:53:41,784 DEBUG [StoreOpener-ebab26be0da503a9bd5db0a862dc884b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/hbase/namespace/ebab26be0da503a9bd5db0a862dc884b/info 2023-02-11 03:53:41,784 INFO [StoreOpener-ebab26be0da503a9bd5db0a862dc884b-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region ebab26be0da503a9bd5db0a862dc884b columnFamilyName info 2023-02-11 03:53:41,785 INFO [StoreOpener-ebab26be0da503a9bd5db0a862dc884b-1] regionserver.HStore(310): Store=ebab26be0da503a9bd5db0a862dc884b/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 03:53:41,787 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/hbase/namespace/ebab26be0da503a9bd5db0a862dc884b 2023-02-11 03:53:41,788 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/hbase/namespace/ebab26be0da503a9bd5db0a862dc884b 2023-02-11 03:53:41,792 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for ebab26be0da503a9bd5db0a862dc884b 2023-02-11 03:53:41,795 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/hbase/namespace/ebab26be0da503a9bd5db0a862dc884b/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-11 03:53:41,796 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened ebab26be0da503a9bd5db0a862dc884b; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=65445381, jitterRate=-0.02478782832622528}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-11 03:53:41,796 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for ebab26be0da503a9bd5db0a862dc884b: Running coprocessor pre-open hook at 1676087621780Writing region info on filesystem at 1676087621780Initializing all the Stores at 1676087621781 (+1 ms)Instantiating store for column family {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} at 1676087621781Cleaning up temporary data from old regions at 1676087621788 (+7 ms)Cleaning up detritus from prior splits at 1676087621789 (+1 ms)Running coprocessor post-open hooks at 1676087621796 (+7 ms)Region opened successfully at 1676087621796 2023-02-11 03:53:41,798 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for hbase:namespace,,1676087620517.ebab26be0da503a9bd5db0a862dc884b., pid=6, masterSystemTime=1676087621771 2023-02-11 03:53:41,802 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for hbase:namespace,,1676087620517.ebab26be0da503a9bd5db0a862dc884b. 2023-02-11 03:53:41,802 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened hbase:namespace,,1676087620517.ebab26be0da503a9bd5db0a862dc884b. 2023-02-11 03:53:41,803 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=ebab26be0da503a9bd5db0a862dc884b, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:53:41,804 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"hbase:namespace,,1676087620517.ebab26be0da503a9bd5db0a862dc884b.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1676087621803"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676087621803"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676087621803"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676087621803"}]},"ts":"1676087621803"} 2023-02-11 03:53:41,810 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=6, resume processing ppid=5 2023-02-11 03:53:41,811 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=6, ppid=5, state=SUCCESS; OpenRegionProcedure ebab26be0da503a9bd5db0a862dc884b, server=jenkins-hbase4.apache.org,37011,1676087618428 in 189 msec 2023-02-11 03:53:41,814 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=5, resume processing ppid=4 2023-02-11 03:53:41,814 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=5, ppid=4, state=SUCCESS; TransitRegionStateProcedure table=hbase:namespace, region=ebab26be0da503a9bd5db0a862dc884b, ASSIGN in 353 msec 2023-02-11 03:53:41,815 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-11 03:53:41,815 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676087621815"}]},"ts":"1676087621815"} 2023-02-11 03:53:41,818 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLED in hbase:meta 2023-02-11 03:53:41,821 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_POST_OPERATION 2023-02-11 03:53:41,824 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace in 1.3030 sec 2023-02-11 03:53:41,829 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/namespace 2023-02-11 03:53:41,831 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/namespace 2023-02-11 03:53:41,831 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-11 03:53:41,863 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=7, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=default 2023-02-11 03:53:41,879 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-02-11 03:53:41,884 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default in 28 msec 2023-02-11 03:53:41,897 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=8, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=hbase 2023-02-11 03:53:41,907 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-02-11 03:53:41,913 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase in 14 msec 2023-02-11 03:53:41,923 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/namespace/default 2023-02-11 03:53:41,927 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/namespace/hbase 2023-02-11 03:53:41,928 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1077): Master has completed initialization 3.430sec 2023-02-11 03:53:41,930 INFO [master/jenkins-hbase4:0:becomeActiveMaster] quotas.MasterQuotaManager(97): Quota support disabled 2023-02-11 03:53:41,931 INFO [master/jenkins-hbase4:0:becomeActiveMaster] slowlog.SlowLogMasterService(57): Slow/Large requests logging to system table hbase:slowlog is disabled. Quitting. 2023-02-11 03:53:41,931 INFO [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKWatcher(269): not a secure deployment, proceeding 2023-02-11 03:53:41,932 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,39921,1676087617326-ExpiredMobFileCleanerChore, period=86400, unit=SECONDS is enabled. 2023-02-11 03:53:41,934 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,39921,1676087617326-MobCompactionChore, period=604800, unit=SECONDS is enabled. 2023-02-11 03:53:41,944 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1166): Balancer post startup initialization complete, took 0 seconds 2023-02-11 03:53:41,969 DEBUG [Listener at localhost/46205] zookeeper.ReadOnlyZKClient(139): Connect 0x561bf69b to 127.0.0.1:53383 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-11 03:53:41,974 DEBUG [Listener at localhost/46205] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@18f0808a, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-11 03:53:41,986 DEBUG [hconnection-0xcc760f5-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-11 03:53:41,997 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:55384, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-11 03:53:42,007 INFO [Listener at localhost/46205] hbase.HBaseTestingUtility(1145): Minicluster is up; activeMaster=jenkins-hbase4.apache.org,39921,1676087617326 2023-02-11 03:53:42,008 DEBUG [Listener at localhost/46205] zookeeper.ReadOnlyZKClient(139): Connect 0x6b586bb5 to 127.0.0.1:53383 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-11 03:53:42,012 DEBUG [Listener at localhost/46205] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@69d78ff9, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-11 03:53:42,013 INFO [Listener at localhost/46205] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-11 03:53:42,033 INFO [Listener at localhost/46205] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMajorCompactionWithWeeklyFollowedByMonthly Thread=210, OpenFileDescriptor=475, MaxFileDescriptor=60000, SystemLoadAverage=252, ProcessCount=168, AvailableMemoryMB=4084 2023-02-11 03:53:42,044 DEBUG [Listener at localhost/46205] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-02-11 03:53:42,047 INFO [RS-EventLoopGroup-1-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:43968, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-02-11 03:53:42,059 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] master.HMaster$4(2102): Client=jenkins//172.31.14.131 create 'testMajorCompactionWithWeeklyFollowedByMonthly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} 2023-02-11 03:53:42,061 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] procedure2.ProcedureExecutor(1029): Stored pid=9, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:42,062 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly execute state=CREATE_TABLE_PRE_OPERATION 2023-02-11 03:53:42,064 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-11 03:53:42,066 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] master.MasterRpcServices(697): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testMajorCompactionWithWeeklyFollowedByMonthly" procId is: 9 2023-02-11 03:53:42,067 DEBUG [HFileArchiver-2] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/aa18866afc8385f0b9c9afab2244ee1e 2023-02-11 03:53:42,068 DEBUG [HFileArchiver-2] backup.HFileArchiver(153): Directory hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/aa18866afc8385f0b9c9afab2244ee1e empty. 2023-02-11 03:53:42,070 DEBUG [HFileArchiver-2] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/aa18866afc8385f0b9c9afab2244ee1e 2023-02-11 03:53:42,070 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testMajorCompactionWithWeeklyFollowedByMonthly regions 2023-02-11 03:53:42,077 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] master.MasterRpcServices(1227): Checking to see if procedure is done pid=9 2023-02-11 03:53:42,182 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] master.MasterRpcServices(1227): Checking to see if procedure is done pid=9 2023-02-11 03:53:42,384 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] master.MasterRpcServices(1227): Checking to see if procedure is done pid=9 2023-02-11 03:53:42,489 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/.tabledesc/.tableinfo.0000000001 2023-02-11 03:53:42,491 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(7671): creating {ENCODED => aa18866afc8385f0b9c9afab2244ee1e, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthly,,1676087622055.aa18866afc8385f0b9c9afab2244ee1e.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testMajorCompactionWithWeeklyFollowedByMonthly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}, regionDir=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/.tmp 2023-02-11 03:53:42,567 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(865): Instantiated testMajorCompactionWithWeeklyFollowedByMonthly,,1676087622055.aa18866afc8385f0b9c9afab2244ee1e.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-11 03:53:42,567 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(1603): Closing aa18866afc8385f0b9c9afab2244ee1e, disabling compactions & flushes 2023-02-11 03:53:42,567 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(1625): Closing region testMajorCompactionWithWeeklyFollowedByMonthly,,1676087622055.aa18866afc8385f0b9c9afab2244ee1e. 2023-02-11 03:53:42,568 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthly,,1676087622055.aa18866afc8385f0b9c9afab2244ee1e. 2023-02-11 03:53:42,568 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(1713): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthly,,1676087622055.aa18866afc8385f0b9c9afab2244ee1e. after waiting 0 ms 2023-02-11 03:53:42,568 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(1723): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthly,,1676087622055.aa18866afc8385f0b9c9afab2244ee1e. 2023-02-11 03:53:42,568 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(1837): Closed testMajorCompactionWithWeeklyFollowedByMonthly,,1676087622055.aa18866afc8385f0b9c9afab2244ee1e. 2023-02-11 03:53:42,568 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(1557): Region close journal for aa18866afc8385f0b9c9afab2244ee1e: Waiting for close lock at 1676087622567Disabling compacts and flushes for region at 1676087622567Disabling writes for close at 1676087622568 (+1 ms)Writing region close event to WAL at 1676087622568Closed at 1676087622568 2023-02-11 03:53:42,572 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly execute state=CREATE_TABLE_ADD_TO_META 2023-02-11 03:53:42,574 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyFollowedByMonthly,,1676087622055.aa18866afc8385f0b9c9afab2244ee1e.","families":{"info":[{"qualifier":"regioninfo","vlen":80,"tag":[],"timestamp":"1676087622574"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676087622574"}]},"ts":"1676087622574"} 2023-02-11 03:53:42,577 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-11 03:53:42,578 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-11 03:53:42,578 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithWeeklyFollowedByMonthly","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676087622578"}]},"ts":"1676087622578"} 2023-02-11 03:53:42,581 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithWeeklyFollowedByMonthly, state=ENABLING in hbase:meta 2023-02-11 03:53:42,686 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] master.MasterRpcServices(1227): Checking to see if procedure is done pid=9 2023-02-11 03:53:42,689 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly, region=aa18866afc8385f0b9c9afab2244ee1e, ASSIGN}] 2023-02-11 03:53:42,691 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly, region=aa18866afc8385f0b9c9afab2244ee1e, ASSIGN 2023-02-11 03:53:42,693 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly, region=aa18866afc8385f0b9c9afab2244ee1e, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,37011,1676087618428; forceNewPlan=false, retain=false 2023-02-11 03:53:42,844 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=10 updating hbase:meta row=aa18866afc8385f0b9c9afab2244ee1e, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:53:42,844 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthly,,1676087622055.aa18866afc8385f0b9c9afab2244ee1e.","families":{"info":[{"qualifier":"regioninfo","vlen":80,"tag":[],"timestamp":"1676087622844"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676087622844"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676087622844"}]},"ts":"1676087622844"} 2023-02-11 03:53:42,847 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=11, ppid=10, state=RUNNABLE; OpenRegionProcedure aa18866afc8385f0b9c9afab2244ee1e, server=jenkins-hbase4.apache.org,37011,1676087618428}] 2023-02-11 03:53:43,006 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionWithWeeklyFollowedByMonthly,,1676087622055.aa18866afc8385f0b9c9afab2244ee1e. 2023-02-11 03:53:43,006 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => aa18866afc8385f0b9c9afab2244ee1e, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthly,,1676087622055.aa18866afc8385f0b9c9afab2244ee1e.', STARTKEY => '', ENDKEY => ''} 2023-02-11 03:53:43,007 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyFollowedByMonthly aa18866afc8385f0b9c9afab2244ee1e 2023-02-11 03:53:43,007 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testMajorCompactionWithWeeklyFollowedByMonthly,,1676087622055.aa18866afc8385f0b9c9afab2244ee1e.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-11 03:53:43,007 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for aa18866afc8385f0b9c9afab2244ee1e 2023-02-11 03:53:43,007 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for aa18866afc8385f0b9c9afab2244ee1e 2023-02-11 03:53:43,010 INFO [StoreOpener-aa18866afc8385f0b9c9afab2244ee1e-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region aa18866afc8385f0b9c9afab2244ee1e 2023-02-11 03:53:43,012 DEBUG [StoreOpener-aa18866afc8385f0b9c9afab2244ee1e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/aa18866afc8385f0b9c9afab2244ee1e/family1 2023-02-11 03:53:43,012 DEBUG [StoreOpener-aa18866afc8385f0b9c9afab2244ee1e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/aa18866afc8385f0b9c9afab2244ee1e/family1 2023-02-11 03:53:43,015 INFO [StoreOpener-aa18866afc8385f0b9c9afab2244ee1e-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region aa18866afc8385f0b9c9afab2244ee1e columnFamilyName family1 2023-02-11 03:53:43,018 INFO [StoreOpener-aa18866afc8385f0b9c9afab2244ee1e-1] regionserver.HStore(310): Store=aa18866afc8385f0b9c9afab2244ee1e/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 03:53:43,021 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/aa18866afc8385f0b9c9afab2244ee1e 2023-02-11 03:53:43,022 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/aa18866afc8385f0b9c9afab2244ee1e 2023-02-11 03:53:43,027 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for aa18866afc8385f0b9c9afab2244ee1e 2023-02-11 03:53:43,030 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/aa18866afc8385f0b9c9afab2244ee1e/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-11 03:53:43,031 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened aa18866afc8385f0b9c9afab2244ee1e; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=67228852, jitterRate=0.0017879605293273926}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-11 03:53:43,031 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for aa18866afc8385f0b9c9afab2244ee1e: Running coprocessor pre-open hook at 1676087623007Writing region info on filesystem at 1676087623007Initializing all the Stores at 1676087623008 (+1 ms)Instantiating store for column family {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} at 1676087623009 (+1 ms)Cleaning up temporary data from old regions at 1676087623022 (+13 ms)Cleaning up detritus from prior splits at 1676087623023 (+1 ms)Running coprocessor post-open hooks at 1676087623031 (+8 ms)Region opened successfully at 1676087623031 2023-02-11 03:53:43,032 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionWithWeeklyFollowedByMonthly,,1676087622055.aa18866afc8385f0b9c9afab2244ee1e., pid=11, masterSystemTime=1676087623000 2023-02-11 03:53:43,035 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionWithWeeklyFollowedByMonthly,,1676087622055.aa18866afc8385f0b9c9afab2244ee1e. 2023-02-11 03:53:43,036 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionWithWeeklyFollowedByMonthly,,1676087622055.aa18866afc8385f0b9c9afab2244ee1e. 2023-02-11 03:53:43,036 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=10 updating hbase:meta row=aa18866afc8385f0b9c9afab2244ee1e, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:53:43,037 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyFollowedByMonthly,,1676087622055.aa18866afc8385f0b9c9afab2244ee1e.","families":{"info":[{"qualifier":"regioninfo","vlen":80,"tag":[],"timestamp":"1676087623036"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676087623036"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676087623036"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676087623036"}]},"ts":"1676087623036"} 2023-02-11 03:53:43,042 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=11, resume processing ppid=10 2023-02-11 03:53:43,042 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=11, ppid=10, state=SUCCESS; OpenRegionProcedure aa18866afc8385f0b9c9afab2244ee1e, server=jenkins-hbase4.apache.org,37011,1676087618428 in 193 msec 2023-02-11 03:53:43,045 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=10, resume processing ppid=9 2023-02-11 03:53:43,046 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=10, ppid=9, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly, region=aa18866afc8385f0b9c9afab2244ee1e, ASSIGN in 353 msec 2023-02-11 03:53:43,047 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-11 03:53:43,047 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithWeeklyFollowedByMonthly","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676087623047"}]},"ts":"1676087623047"} 2023-02-11 03:53:43,049 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithWeeklyFollowedByMonthly, state=ENABLED in hbase:meta 2023-02-11 03:53:43,052 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly execute state=CREATE_TABLE_POST_OPERATION 2023-02-11 03:53:43,054 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=9, state=SUCCESS; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly in 993 msec 2023-02-11 03:53:43,188 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] master.MasterRpcServices(1227): Checking to see if procedure is done pid=9 2023-02-11 03:53:43,188 INFO [Listener at localhost/46205] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testMajorCompactionWithWeeklyFollowedByMonthly, procId: 9 completed 2023-02-11 03:53:43,205 DEBUG [hconnection-0x7aa6bd5c-metaLookup-shared--pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-11 03:53:43,208 INFO [RS-EventLoopGroup-3-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:55394, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-11 03:53:43,349 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 03:53:43,357 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:43,357 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-02-11 03:53:43,357 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:53:43,358 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-02-11 03:53:43,358 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-02-11 03:53:43,358 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:43,359 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:43,360 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:53:43,360 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:53:43,360 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:53:43,360 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:53:43,360 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:53:43,360 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 03:53:43,360 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:43,361 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:43,361 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 03:53:43,361 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:43,362 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:43,364 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:43,364 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-02-11 03:53:43,364 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:53:43,365 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-02-11 03:53:43,365 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 03:53:43,365 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 03:53:43,366 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1676087622055.aa18866afc8385f0b9c9afab2244ee1e. 2023-02-11 03:53:43,366 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1676087622055.aa18866afc8385f0b9c9afab2244ee1e. started... 2023-02-11 03:53:43,367 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing aa18866afc8385f0b9c9afab2244ee1e 1/1 column families, dataSize=74.11 KB heapSize=148.69 KB 2023-02-11 03:53:43,912 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151130da6aae65587c4bafb955b834ccd75afc to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151130da6aae65587c4bafb955b834ccd75afc 2023-02-11 03:53:43,928 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=74.1 K, hasBloomFilter=true, into tmp file hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/aa18866afc8385f0b9c9afab2244ee1e/.tmp/family1/7ce3a1bf235e4eb29c256807a397a523 2023-02-11 03:53:43,939 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/aa18866afc8385f0b9c9afab2244ee1e/.tmp/family1/7ce3a1bf235e4eb29c256807a397a523 as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/aa18866afc8385f0b9c9afab2244ee1e/family1/7ce3a1bf235e4eb29c256807a397a523 2023-02-11 03:53:43,948 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/aa18866afc8385f0b9c9afab2244ee1e/family1/7ce3a1bf235e4eb29c256807a397a523, entries=1000, sequenceid=5, filesize=175.8 K 2023-02-11 03:53:43,950 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~74.11 KB/75890, heapSize ~148.67 KB/152240, currentSize=0 B/0 for aa18866afc8385f0b9c9afab2244ee1e in 583ms, sequenceid=5, compaction requested=false 2023-02-11 03:53:43,952 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-02-11 03:53:43,954 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for aa18866afc8385f0b9c9afab2244ee1e: 2023-02-11 03:53:43,954 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1676087622055.aa18866afc8385f0b9c9afab2244ee1e. 2023-02-11 03:53:43,955 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 03:53:43,956 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 03:53:43,956 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:53:43,956 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-02-11 03:53:43,956 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37011,1676087618428' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-02-11 03:53:43,959 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:53:43,959 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:43,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:53:43,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:53:43,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:53:43,959 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:43,959 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 03:53:43,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:53:43,960 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:53:43,960 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:43,961 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:53:43,961 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:53:43,962 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37011,1676087618428' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-02-11 03:53:43,962 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-02-11 03:53:43,962 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7299b7fb[Count = 0] remaining members to acquire global barrier 2023-02-11 03:53:43,962 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:43,965 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:43,965 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:43,965 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:43,965 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-02-11 03:53:43,965 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-02-11 03:53:43,965 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:53:43,965 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase4.apache.org,37011,1676087618428' in zk 2023-02-11 03:53:43,965 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 03:53:43,967 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:53:43,967 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-02-11 03:53:43,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:53:43,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:53:43,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:53:43,967 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:53:43,968 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-02-11 03:53:43,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:53:43,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:53:43,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:43,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:53:43,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:53:43,970 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:43,970 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:53:43,971 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase4.apache.org,37011,1676087618428': 2023-02-11 03:53:43,971 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37011,1676087618428' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-02-11 03:53:43,971 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-02-11 03:53:43,971 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 03:53:43,971 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 03:53:43,971 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:43,971 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 03:53:43,973 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:43,973 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:43,973 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:43,974 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:53:43,974 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:53:43,974 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:43,974 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:53:43,974 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:43,974 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:53:43,974 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:53:43,974 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:53:43,974 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:53:43,974 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:43,975 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:43,975 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:53:43,975 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:43,976 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:53:43,976 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:53:43,976 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:53:43,977 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:43,977 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:53:43,983 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:53:43,983 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:43,983 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:53:43,983 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:43,983 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:53:43,983 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:53:43,983 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:53:43,983 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:43,983 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-02-11 03:53:43,983 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:53:43,983 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 03:53:43,983 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:43,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:53:43,984 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:43,984 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:53:43,984 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:53:43,985 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-02-11 03:53:43,986 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 03:53:44,086 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 03:53:44,091 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 03:53:44,101 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 03:53:44,104 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-02-11 03:53:44,104 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:44,104 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-02-11 03:53:44,104 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:53:44,105 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-02-11 03:53:44,105 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-02-11 03:53:44,105 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:44,105 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:44,112 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:53:44,112 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:53:44,112 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:53:44,113 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:53:44,113 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:53:44,113 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 03:53:44,113 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:44,114 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:44,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 03:53:44,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:44,114 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:44,115 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-02-11 03:53:44,115 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:44,115 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-02-11 03:53:44,115 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:53:44,115 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-02-11 03:53:44,115 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 03:53:44,116 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 03:53:44,116 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1676087622055.aa18866afc8385f0b9c9afab2244ee1e. 2023-02-11 03:53:44,116 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1676087622055.aa18866afc8385f0b9c9afab2244ee1e. started... 2023-02-11 03:53:44,116 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing aa18866afc8385f0b9c9afab2244ee1e 1/1 column families, dataSize=74 B heapSize=408 B 2023-02-11 03:53:44,156 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015112854b80465a8a6469b83b3c750c028fc65 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e2015112854b80465a8a6469b83b3c750c028fc65 2023-02-11 03:53:44,575 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=74, hasBloomFilter=true, into tmp file hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/aa18866afc8385f0b9c9afab2244ee1e/.tmp/family1/2137b3c52d8641c39cd08ea5024bbed9 2023-02-11 03:53:44,584 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/aa18866afc8385f0b9c9afab2244ee1e/.tmp/family1/2137b3c52d8641c39cd08ea5024bbed9 as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/aa18866afc8385f0b9c9afab2244ee1e/family1/2137b3c52d8641c39cd08ea5024bbed9 2023-02-11 03:53:44,593 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/aa18866afc8385f0b9c9afab2244ee1e/family1/2137b3c52d8641c39cd08ea5024bbed9, entries=1, sequenceid=9, filesize=5.0 K 2023-02-11 03:53:44,595 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~74 B/74, heapSize ~392 B/392, currentSize=0 B/0 for aa18866afc8385f0b9c9afab2244ee1e in 479ms, sequenceid=9, compaction requested=false 2023-02-11 03:53:44,595 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for aa18866afc8385f0b9c9afab2244ee1e: 2023-02-11 03:53:44,595 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1676087622055.aa18866afc8385f0b9c9afab2244ee1e. 2023-02-11 03:53:44,595 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 03:53:44,596 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 03:53:44,596 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:53:44,596 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-02-11 03:53:44,596 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37011,1676087618428' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-02-11 03:53:44,598 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:53:44,598 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:44,598 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:53:44,598 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:53:44,598 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:53:44,599 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:44,599 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 03:53:44,599 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:53:44,599 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:53:44,600 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:44,600 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:53:44,600 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:53:44,601 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37011,1676087618428' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-02-11 03:53:44,601 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@11ab18a0[Count = 0] remaining members to acquire global barrier 2023-02-11 03:53:44,601 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-02-11 03:53:44,601 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:44,602 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:44,602 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:44,602 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:44,602 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-02-11 03:53:44,603 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-02-11 03:53:44,603 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase4.apache.org,37011,1676087618428' in zk 2023-02-11 03:53:44,603 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:53:44,603 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 03:53:44,604 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-02-11 03:53:44,604 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:53:44,604 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:53:44,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:53:44,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:53:44,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:53:44,605 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-02-11 03:53:44,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:53:44,606 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:53:44,606 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:44,606 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:53:44,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:53:44,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:44,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:53:44,608 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase4.apache.org,37011,1676087618428': 2023-02-11 03:53:44,608 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37011,1676087618428' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-02-11 03:53:44,608 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-02-11 03:53:44,608 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 03:53:44,608 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 03:53:44,608 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:44,608 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 03:53:44,618 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:44,618 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:44,618 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:44,618 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:53:44,618 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:53:44,618 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:44,618 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:44,618 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:53:44,619 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:53:44,619 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:53:44,619 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:53:44,619 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:53:44,619 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:44,619 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:44,620 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:53:44,620 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:44,621 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:53:44,621 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:53:44,621 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:53:44,622 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:44,622 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:53:44,624 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:53:44,624 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:53:44,624 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:44,624 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:53:44,624 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:53:44,624 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly because node does not exist (not an error) 2023-02-11 03:53:44,624 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-02-11 03:53:44,625 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 03:53:44,624 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:53:44,624 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:44,624 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:53:44,625 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:53:44,625 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:44,625 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:44,625 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:44,625 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-02-11 03:53:44,626 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 03:53:44,626 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:53:44,626 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:53:44,726 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 03:53:44,727 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 03:53:44,737 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 03:53:44,741 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-02-11 03:53:44,741 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:44,741 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-02-11 03:53:44,742 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:53:44,742 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-02-11 03:53:44,742 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-02-11 03:53:44,743 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:44,743 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:44,744 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:53:44,744 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:53:44,745 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:53:44,745 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:53:44,745 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:53:44,745 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 03:53:44,745 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:44,746 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:44,746 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 03:53:44,746 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:44,746 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:44,746 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-02-11 03:53:44,746 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:44,747 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-02-11 03:53:44,747 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:53:44,747 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-02-11 03:53:44,747 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 03:53:44,747 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 03:53:44,747 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1676087622055.aa18866afc8385f0b9c9afab2244ee1e. 2023-02-11 03:53:44,748 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1676087622055.aa18866afc8385f0b9c9afab2244ee1e. started... 2023-02-11 03:53:44,748 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing aa18866afc8385f0b9c9afab2244ee1e 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-11 03:53:45,180 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512016848f71c04514a18b9891bcd2b51ffe6 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201512016848f71c04514a18b9891bcd2b51ffe6 2023-02-11 03:53:45,191 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/aa18866afc8385f0b9c9afab2244ee1e/.tmp/family1/68470dbad13e4b159f7094730288d5b8 2023-02-11 03:53:45,199 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/aa18866afc8385f0b9c9afab2244ee1e/.tmp/family1/68470dbad13e4b159f7094730288d5b8 as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/aa18866afc8385f0b9c9afab2244ee1e/family1/68470dbad13e4b159f7094730288d5b8 2023-02-11 03:53:45,207 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/aa18866afc8385f0b9c9afab2244ee1e/family1/68470dbad13e4b159f7094730288d5b8, entries=1, sequenceid=13, filesize=5.0 K 2023-02-11 03:53:45,208 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for aa18866afc8385f0b9c9afab2244ee1e in 460ms, sequenceid=13, compaction requested=true 2023-02-11 03:53:45,209 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for aa18866afc8385f0b9c9afab2244ee1e: 2023-02-11 03:53:45,209 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1676087622055.aa18866afc8385f0b9c9afab2244ee1e. 2023-02-11 03:53:45,209 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 03:53:45,209 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 03:53:45,209 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:53:45,209 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-02-11 03:53:45,209 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37011,1676087618428' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-02-11 03:53:45,212 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:53:45,212 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:45,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:53:45,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:53:45,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:53:45,213 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:45,213 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 03:53:45,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:53:45,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:53:45,214 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:45,214 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:53:45,215 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:53:45,215 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37011,1676087618428' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-02-11 03:53:45,215 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1d631831[Count = 0] remaining members to acquire global barrier 2023-02-11 03:53:45,215 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-02-11 03:53:45,215 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:45,217 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:45,217 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:45,217 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:45,217 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-02-11 03:53:45,217 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-02-11 03:53:45,217 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase4.apache.org,37011,1676087618428' in zk 2023-02-11 03:53:45,217 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:53:45,217 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 03:53:45,219 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-02-11 03:53:45,219 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:53:45,219 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:53:45,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:53:45,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:53:45,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:53:45,219 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-02-11 03:53:45,220 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:53:45,220 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:53:45,221 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:45,221 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:53:45,221 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:53:45,222 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:45,222 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:53:45,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase4.apache.org,37011,1676087618428': 2023-02-11 03:53:45,223 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37011,1676087618428' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-02-11 03:53:45,223 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-02-11 03:53:45,223 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 03:53:45,223 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 03:53:45,223 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:45,223 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 03:53:45,232 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:45,232 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:45,232 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:53:45,232 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:45,232 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:53:45,232 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:53:45,232 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:45,232 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:45,233 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:53:45,233 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:53:45,233 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:53:45,233 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:53:45,233 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:45,234 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:45,235 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:53:45,235 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:45,236 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:53:45,236 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:53:45,236 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:53:45,237 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:45,238 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:53:45,242 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:53:45,242 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:53:45,242 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:53:45,242 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:53:45,242 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:45,242 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:45,242 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:53:45,243 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:45,243 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:45,243 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:45,242 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:53:45,243 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly because node does not exist (not an error) 2023-02-11 03:53:45,242 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-02-11 03:53:45,243 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 03:53:45,242 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:53:45,248 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-02-11 03:53:45,249 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:53:45,249 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:53:45,249 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 03:53:45,349 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 03:53:45,351 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 03:53:45,360 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 03:53:45,362 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-02-11 03:53:45,363 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:45,363 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-02-11 03:53:45,363 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:53:45,363 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-02-11 03:53:45,363 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-02-11 03:53:45,364 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:45,364 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:45,365 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:53:45,365 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:53:45,365 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:53:45,365 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:53:45,366 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:53:45,366 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 03:53:45,366 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:45,366 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:45,367 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 03:53:45,367 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:45,367 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:45,367 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-02-11 03:53:45,367 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:45,367 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-02-11 03:53:45,367 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:53:45,368 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-02-11 03:53:45,368 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 03:53:45,368 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 03:53:45,368 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1676087622055.aa18866afc8385f0b9c9afab2244ee1e. 2023-02-11 03:53:45,368 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1676087622055.aa18866afc8385f0b9c9afab2244ee1e. started... 2023-02-11 03:53:45,368 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing aa18866afc8385f0b9c9afab2244ee1e 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-11 03:53:45,797 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151205895aab49be144fd6b4e1f5112c1ddc09 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151205895aab49be144fd6b4e1f5112c1ddc09 2023-02-11 03:53:45,839 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-11 03:53:45,953 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:meta' 2023-02-11 03:53:45,955 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:namespace' 2023-02-11 03:53:46,215 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/aa18866afc8385f0b9c9afab2244ee1e/.tmp/family1/ba8f9ade0aeb4025827d6678e12285fa 2023-02-11 03:53:46,228 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/aa18866afc8385f0b9c9afab2244ee1e/.tmp/family1/ba8f9ade0aeb4025827d6678e12285fa as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/aa18866afc8385f0b9c9afab2244ee1e/family1/ba8f9ade0aeb4025827d6678e12285fa 2023-02-11 03:53:46,238 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/aa18866afc8385f0b9c9afab2244ee1e/family1/ba8f9ade0aeb4025827d6678e12285fa, entries=1, sequenceid=17, filesize=5.0 K 2023-02-11 03:53:46,240 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for aa18866afc8385f0b9c9afab2244ee1e in 872ms, sequenceid=17, compaction requested=true 2023-02-11 03:53:46,240 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for aa18866afc8385f0b9c9afab2244ee1e: 2023-02-11 03:53:46,240 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1676087622055.aa18866afc8385f0b9c9afab2244ee1e. 2023-02-11 03:53:46,240 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 03:53:46,241 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 03:53:46,241 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:53:46,241 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-02-11 03:53:46,241 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37011,1676087618428' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-02-11 03:53:46,243 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:53:46,243 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:46,243 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:53:46,244 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:53:46,244 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:53:46,244 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:46,244 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 03:53:46,244 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:53:46,245 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:53:46,245 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:46,245 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:53:46,246 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:53:46,246 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37011,1676087618428' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-02-11 03:53:46,246 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4bc1ba09[Count = 0] remaining members to acquire global barrier 2023-02-11 03:53:46,246 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-02-11 03:53:46,246 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:46,248 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:46,248 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:46,248 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:46,248 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-02-11 03:53:46,248 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-02-11 03:53:46,248 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase4.apache.org,37011,1676087618428' in zk 2023-02-11 03:53:46,248 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:53:46,248 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 03:53:46,251 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-02-11 03:53:46,251 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:53:46,251 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:53:46,251 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:53:46,251 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:53:46,251 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:53:46,251 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-02-11 03:53:46,252 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:53:46,252 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:53:46,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:46,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:53:46,254 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:53:46,254 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:46,255 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:53:46,255 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase4.apache.org,37011,1676087618428': 2023-02-11 03:53:46,256 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37011,1676087618428' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-02-11 03:53:46,256 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-02-11 03:53:46,256 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 03:53:46,256 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 03:53:46,256 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:46,256 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 03:53:46,258 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:46,258 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:53:46,258 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:46,258 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:46,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:46,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:53:46,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:53:46,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:46,259 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:53:46,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:53:46,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:53:46,259 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:53:46,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:46,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:46,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:53:46,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:46,261 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:53:46,262 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:53:46,262 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:53:46,263 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:46,263 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:53:46,267 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:53:46,267 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:53:46,267 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:53:46,267 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:53:46,267 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:53:46,267 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:46,267 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:46,267 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:53:46,267 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:53:46,268 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:53:46,268 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:46,268 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:46,268 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:46,268 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:53:46,268 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-02-11 03:53:46,269 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 03:53:46,269 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly because node does not exist (not an error) 2023-02-11 03:53:46,269 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-02-11 03:53:46,269 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 03:53:46,370 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 03:53:46,372 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 03:53:46,380 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 03:53:46,383 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-02-11 03:53:46,384 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:46,384 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-02-11 03:53:46,384 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:53:46,384 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-02-11 03:53:46,384 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-02-11 03:53:46,385 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:46,385 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:46,387 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:53:46,387 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:53:46,387 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:53:46,387 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:53:46,387 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:53:46,387 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 03:53:46,387 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:46,388 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:46,388 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 03:53:46,388 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:46,388 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:46,389 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-02-11 03:53:46,389 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:46,389 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-02-11 03:53:46,389 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:53:46,389 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-02-11 03:53:46,390 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 03:53:46,390 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 03:53:46,390 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1676087622055.aa18866afc8385f0b9c9afab2244ee1e. 2023-02-11 03:53:46,390 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1676087622055.aa18866afc8385f0b9c9afab2244ee1e. started... 2023-02-11 03:53:46,390 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing aa18866afc8385f0b9c9afab2244ee1e 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-11 03:53:46,819 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512286bfb9c6fdf6b46c6bedbcaaded338cdc to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201512286bfb9c6fdf6b46c6bedbcaaded338cdc 2023-02-11 03:53:47,230 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=21, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/aa18866afc8385f0b9c9afab2244ee1e/.tmp/family1/9a8fd510cb944335a1130636196e0b6f 2023-02-11 03:53:47,238 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/aa18866afc8385f0b9c9afab2244ee1e/.tmp/family1/9a8fd510cb944335a1130636196e0b6f as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/aa18866afc8385f0b9c9afab2244ee1e/family1/9a8fd510cb944335a1130636196e0b6f 2023-02-11 03:53:47,246 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/aa18866afc8385f0b9c9afab2244ee1e/family1/9a8fd510cb944335a1130636196e0b6f, entries=1, sequenceid=21, filesize=5.0 K 2023-02-11 03:53:47,247 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for aa18866afc8385f0b9c9afab2244ee1e in 857ms, sequenceid=21, compaction requested=true 2023-02-11 03:53:47,247 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for aa18866afc8385f0b9c9afab2244ee1e: 2023-02-11 03:53:47,247 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1676087622055.aa18866afc8385f0b9c9afab2244ee1e. 2023-02-11 03:53:47,247 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 03:53:47,248 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 03:53:47,248 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:53:47,248 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-02-11 03:53:47,248 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37011,1676087618428' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-02-11 03:53:47,254 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:53:47,254 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:47,254 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:53:47,254 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:53:47,254 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:53:47,254 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:47,254 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 03:53:47,255 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:53:47,255 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:53:47,255 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:47,255 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:53:47,256 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:53:47,256 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37011,1676087618428' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-02-11 03:53:47,256 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@74d8fae5[Count = 0] remaining members to acquire global barrier 2023-02-11 03:53:47,256 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-02-11 03:53:47,256 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:47,257 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:47,257 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:47,257 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:47,257 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-02-11 03:53:47,257 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-02-11 03:53:47,258 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase4.apache.org,37011,1676087618428' in zk 2023-02-11 03:53:47,258 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:53:47,258 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 03:53:47,259 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:53:47,259 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-02-11 03:53:47,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:53:47,259 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:53:47,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:53:47,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:53:47,259 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-02-11 03:53:47,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:53:47,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:53:47,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:47,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:53:47,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:53:47,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:47,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:53:47,262 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase4.apache.org,37011,1676087618428': 2023-02-11 03:53:47,262 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37011,1676087618428' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-02-11 03:53:47,262 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-02-11 03:53:47,262 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 03:53:47,262 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 03:53:47,262 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:47,262 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 03:53:47,264 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:47,264 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:47,264 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:47,264 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:47,264 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:53:47,264 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:53:47,264 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:53:47,264 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:47,265 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:53:47,265 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:53:47,265 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:53:47,265 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:53:47,265 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:47,265 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:47,265 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:53:47,266 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:47,266 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:53:47,266 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:53:47,267 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:53:47,267 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:47,267 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:53:47,270 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:53:47,270 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:53:47,270 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:47,270 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:53:47,270 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:53:47,270 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:53:47,270 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-02-11 03:53:47,270 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 03:53:47,270 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:53:47,270 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:47,270 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly because node does not exist (not an error) 2023-02-11 03:53:47,271 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:53:47,271 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:53:47,271 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:53:47,271 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:47,271 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:47,271 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-02-11 03:53:47,271 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:47,271 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 03:53:47,371 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 03:53:47,373 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 03:53:47,380 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 03:53:47,383 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-02-11 03:53:47,383 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:47,383 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-02-11 03:53:47,383 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:53:47,383 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-02-11 03:53:47,384 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-02-11 03:53:47,384 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:47,384 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:47,385 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:53:47,385 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:53:47,385 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:53:47,385 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:53:47,386 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:53:47,386 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 03:53:47,386 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:47,386 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:47,387 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 03:53:47,387 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:47,387 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:47,387 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-02-11 03:53:47,387 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:47,387 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-02-11 03:53:47,387 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:53:47,387 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-02-11 03:53:47,388 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 03:53:47,388 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 03:53:47,388 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1676087622055.aa18866afc8385f0b9c9afab2244ee1e. 2023-02-11 03:53:47,388 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1676087622055.aa18866afc8385f0b9c9afab2244ee1e. started... 2023-02-11 03:53:47,388 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing aa18866afc8385f0b9c9afab2244ee1e 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-11 03:53:47,475 DEBUG [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(130): Registering adapter for the MetricRegistry: RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-02-11 03:53:47,476 INFO [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(134): Registering RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint Metrics about HBase RegionObservers 2023-02-11 03:53:47,810 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151231ed2ec05b3b78468f98c65b65ee206b78 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151231ed2ec05b3b78468f98c65b65ee206b78 2023-02-11 03:53:47,821 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=25, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/aa18866afc8385f0b9c9afab2244ee1e/.tmp/family1/8b1f001b1b1343cea954814d01294a86 2023-02-11 03:53:47,829 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/aa18866afc8385f0b9c9afab2244ee1e/.tmp/family1/8b1f001b1b1343cea954814d01294a86 as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/aa18866afc8385f0b9c9afab2244ee1e/family1/8b1f001b1b1343cea954814d01294a86 2023-02-11 03:53:47,836 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/aa18866afc8385f0b9c9afab2244ee1e/family1/8b1f001b1b1343cea954814d01294a86, entries=1, sequenceid=25, filesize=5.0 K 2023-02-11 03:53:47,837 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for aa18866afc8385f0b9c9afab2244ee1e in 449ms, sequenceid=25, compaction requested=true 2023-02-11 03:53:47,838 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for aa18866afc8385f0b9c9afab2244ee1e: 2023-02-11 03:53:47,838 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1676087622055.aa18866afc8385f0b9c9afab2244ee1e. 2023-02-11 03:53:47,838 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 03:53:47,838 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 03:53:47,839 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:53:47,839 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-02-11 03:53:47,839 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37011,1676087618428' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-02-11 03:53:47,840 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:47,840 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:53:47,841 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:53:47,841 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:53:47,841 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:53:47,841 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:47,841 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 03:53:47,841 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:53:47,841 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:53:47,842 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:47,842 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:53:47,842 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:53:47,843 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37011,1676087618428' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-02-11 03:53:47,843 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3d7f82c6[Count = 0] remaining members to acquire global barrier 2023-02-11 03:53:47,843 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-02-11 03:53:47,843 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:47,844 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:47,844 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:47,844 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:47,844 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-02-11 03:53:47,844 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-02-11 03:53:47,844 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase4.apache.org,37011,1676087618428' in zk 2023-02-11 03:53:47,844 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:53:47,844 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 03:53:47,846 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:53:47,846 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-02-11 03:53:47,846 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:53:47,846 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:53:47,846 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:53:47,846 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:53:47,846 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-02-11 03:53:47,847 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:53:47,847 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:53:47,847 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:47,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:53:47,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:53:47,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:47,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:53:47,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase4.apache.org,37011,1676087618428': 2023-02-11 03:53:47,849 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37011,1676087618428' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-02-11 03:53:47,849 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-02-11 03:53:47,849 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 03:53:47,849 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 03:53:47,849 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:47,849 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 03:53:47,852 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:47,852 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:47,852 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:47,852 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:47,852 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:53:47,852 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:53:47,852 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:53:47,852 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:47,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:53:47,853 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:53:47,853 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:53:47,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:53:47,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:47,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:47,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:53:47,854 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:47,854 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:53:47,854 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:53:47,855 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:53:47,855 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:47,856 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:53:47,857 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:53:47,857 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:47,857 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:53:47,857 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:47,857 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:53:47,857 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:53:47,857 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:53:47,857 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:47,857 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:53:47,858 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-02-11 03:53:47,858 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 03:53:47,858 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly because node does not exist (not an error) 2023-02-11 03:53:47,858 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:53:47,858 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:47,858 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:47,859 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:53:47,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:53:47,859 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-02-11 03:53:47,859 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 03:53:47,959 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 03:53:47,960 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 03:53:47,968 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 03:53:47,970 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-02-11 03:53:47,970 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:47,970 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-02-11 03:53:47,970 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:53:47,970 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-02-11 03:53:47,970 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-02-11 03:53:47,971 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:47,971 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:47,972 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:53:47,972 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:53:47,972 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:53:47,973 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:53:47,973 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:53:47,973 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 03:53:47,973 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:47,974 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:47,974 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 03:53:47,974 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:47,974 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:47,974 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-02-11 03:53:47,975 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:47,975 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-02-11 03:53:47,975 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:53:47,975 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-02-11 03:53:47,975 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 03:53:47,975 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 03:53:47,975 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1676087622055.aa18866afc8385f0b9c9afab2244ee1e. 2023-02-11 03:53:47,975 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1676087622055.aa18866afc8385f0b9c9afab2244ee1e. started... 2023-02-11 03:53:47,975 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing aa18866afc8385f0b9c9afab2244ee1e 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-11 03:53:48,399 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201601012f7ce5aa547e409bbfb59cf496dc0285 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201601012f7ce5aa547e409bbfb59cf496dc0285 2023-02-11 03:53:48,807 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=29, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/aa18866afc8385f0b9c9afab2244ee1e/.tmp/family1/070b081fe1c64ad081eea06b4e2f0f44 2023-02-11 03:53:48,815 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/aa18866afc8385f0b9c9afab2244ee1e/.tmp/family1/070b081fe1c64ad081eea06b4e2f0f44 as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/aa18866afc8385f0b9c9afab2244ee1e/family1/070b081fe1c64ad081eea06b4e2f0f44 2023-02-11 03:53:48,823 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/aa18866afc8385f0b9c9afab2244ee1e/family1/070b081fe1c64ad081eea06b4e2f0f44, entries=1, sequenceid=29, filesize=5.0 K 2023-02-11 03:53:48,824 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for aa18866afc8385f0b9c9afab2244ee1e in 849ms, sequenceid=29, compaction requested=true 2023-02-11 03:53:48,824 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for aa18866afc8385f0b9c9afab2244ee1e: 2023-02-11 03:53:48,824 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1676087622055.aa18866afc8385f0b9c9afab2244ee1e. 2023-02-11 03:53:48,824 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 03:53:48,825 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 03:53:48,825 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:53:48,825 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-02-11 03:53:48,825 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37011,1676087618428' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-02-11 03:53:48,827 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:53:48,827 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:48,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:53:48,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:53:48,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:53:48,827 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:48,827 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 03:53:48,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:53:48,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:53:48,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:48,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:53:48,829 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:53:48,829 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37011,1676087618428' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-02-11 03:53:48,829 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3858a7c8[Count = 0] remaining members to acquire global barrier 2023-02-11 03:53:48,829 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-02-11 03:53:48,829 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:48,831 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:48,831 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:48,831 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:48,831 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-02-11 03:53:48,831 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-02-11 03:53:48,832 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase4.apache.org,37011,1676087618428' in zk 2023-02-11 03:53:48,832 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:53:48,832 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 03:53:48,833 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-02-11 03:53:48,833 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:53:48,833 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:53:48,833 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:53:48,833 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:53:48,833 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:53:48,833 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-02-11 03:53:48,834 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:53:48,834 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:53:48,834 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:48,834 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:53:48,835 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:53:48,835 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:48,835 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:53:48,836 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase4.apache.org,37011,1676087618428': 2023-02-11 03:53:48,836 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37011,1676087618428' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-02-11 03:53:48,836 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-02-11 03:53:48,836 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 03:53:48,836 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 03:53:48,836 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:48,836 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 03:53:48,837 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:48,838 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:48,838 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:48,838 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:48,838 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:53:48,838 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:53:48,838 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:53:48,838 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:48,838 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:53:48,838 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:53:48,838 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:53:48,838 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:53:48,838 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:48,839 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:48,839 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:53:48,839 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:48,839 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:53:48,839 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:53:48,840 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:53:48,840 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:48,841 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:53:48,845 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:53:48,845 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:48,845 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:53:48,845 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:48,846 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-02-11 03:53:48,846 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 03:53:48,846 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly because node does not exist (not an error) 2023-02-11 03:53:48,846 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:53:48,846 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:53:48,846 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:53:48,846 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:53:48,846 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:53:48,846 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:48,846 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-02-11 03:53:48,847 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 03:53:48,846 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:48,847 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:48,847 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:53:48,847 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:53:48,947 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 03:53:48,948 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 03:53:48,955 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 03:53:48,957 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-02-11 03:53:48,957 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:48,958 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-02-11 03:53:48,958 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:53:48,958 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-02-11 03:53:48,958 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-02-11 03:53:48,958 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:48,959 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:48,960 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:53:48,960 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:53:48,960 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:53:48,960 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:53:48,960 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:53:48,960 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 03:53:48,960 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:48,961 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:48,961 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 03:53:48,961 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:48,961 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:48,961 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-02-11 03:53:48,961 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:48,961 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-02-11 03:53:48,962 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:53:48,962 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-02-11 03:53:48,962 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 03:53:48,962 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 03:53:48,962 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1676087622055.aa18866afc8385f0b9c9afab2244ee1e. 2023-02-11 03:53:48,962 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1676087622055.aa18866afc8385f0b9c9afab2244ee1e. started... 2023-02-11 03:53:48,962 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing aa18866afc8385f0b9c9afab2244ee1e 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-11 03:53:49,383 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2016010378ad5eaaf02d46b4b6bd5bf99a092408 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e2016010378ad5eaaf02d46b4b6bd5bf99a092408 2023-02-11 03:53:49,391 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=33, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/aa18866afc8385f0b9c9afab2244ee1e/.tmp/family1/7f8e9ad2e58346d6af31ab4cea7e13e7 2023-02-11 03:53:49,399 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/aa18866afc8385f0b9c9afab2244ee1e/.tmp/family1/7f8e9ad2e58346d6af31ab4cea7e13e7 as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/aa18866afc8385f0b9c9afab2244ee1e/family1/7f8e9ad2e58346d6af31ab4cea7e13e7 2023-02-11 03:53:49,405 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/aa18866afc8385f0b9c9afab2244ee1e/family1/7f8e9ad2e58346d6af31ab4cea7e13e7, entries=1, sequenceid=33, filesize=5.0 K 2023-02-11 03:53:49,406 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for aa18866afc8385f0b9c9afab2244ee1e in 444ms, sequenceid=33, compaction requested=true 2023-02-11 03:53:49,407 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for aa18866afc8385f0b9c9afab2244ee1e: 2023-02-11 03:53:49,407 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1676087622055.aa18866afc8385f0b9c9afab2244ee1e. 2023-02-11 03:53:49,407 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 03:53:49,407 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 03:53:49,407 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:53:49,407 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-02-11 03:53:49,407 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37011,1676087618428' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-02-11 03:53:49,409 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:49,409 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:53:49,409 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:53:49,409 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:53:49,409 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:53:49,409 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:49,409 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 03:53:49,409 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:53:49,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:53:49,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:49,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:53:49,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:53:49,411 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37011,1676087618428' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-02-11 03:53:49,411 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7e0f4338[Count = 0] remaining members to acquire global barrier 2023-02-11 03:53:49,411 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-02-11 03:53:49,411 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:49,412 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:49,412 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:49,412 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:49,412 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-02-11 03:53:49,412 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-02-11 03:53:49,413 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase4.apache.org,37011,1676087618428' in zk 2023-02-11 03:53:49,413 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:53:49,413 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 03:53:49,415 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-02-11 03:53:49,415 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:53:49,415 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:53:49,415 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:53:49,415 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:53:49,415 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:53:49,415 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-02-11 03:53:49,416 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:53:49,416 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:53:49,416 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:49,417 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:53:49,417 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:53:49,417 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:49,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:53:49,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase4.apache.org,37011,1676087618428': 2023-02-11 03:53:49,418 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37011,1676087618428' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-02-11 03:53:49,418 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-02-11 03:53:49,419 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 03:53:49,419 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 03:53:49,419 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:49,419 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 03:53:49,420 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:49,420 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:49,420 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:49,420 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:49,420 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:53:49,420 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:53:49,420 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:53:49,420 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:49,421 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:53:49,421 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:53:49,421 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:53:49,421 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:53:49,421 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:49,422 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:49,422 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:53:49,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:49,423 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:53:49,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:53:49,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:53:49,424 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:49,424 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:53:49,425 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:53:49,425 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:53:49,425 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:49,425 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:53:49,425 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:53:49,425 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-02-11 03:53:49,426 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 03:53:49,425 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly because node does not exist (not an error) 2023-02-11 03:53:49,425 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:53:49,425 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:49,426 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:53:49,426 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:53:49,426 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:49,426 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:49,426 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-02-11 03:53:49,426 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 03:53:49,426 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:49,427 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:53:49,427 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:53:49,527 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 03:53:49,528 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 03:53:49,535 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 03:53:49,537 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-02-11 03:53:49,537 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:49,537 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-02-11 03:53:49,537 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:53:49,537 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-02-11 03:53:49,538 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-02-11 03:53:49,538 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:49,538 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:49,539 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:53:49,539 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:53:49,539 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:53:49,540 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:53:49,540 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:53:49,540 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 03:53:49,540 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:49,540 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:49,541 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 03:53:49,541 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:49,541 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:49,541 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-02-11 03:53:49,541 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:49,542 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-02-11 03:53:49,542 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:53:49,543 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-02-11 03:53:49,543 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 03:53:49,543 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 03:53:49,543 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1676087622055.aa18866afc8385f0b9c9afab2244ee1e. 2023-02-11 03:53:49,543 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1676087622055.aa18866afc8385f0b9c9afab2244ee1e. started... 2023-02-11 03:53:49,543 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing aa18866afc8385f0b9c9afab2244ee1e 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-11 03:53:49,962 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015090719ae5329ea184b2382507190a6c21e23 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e2015090719ae5329ea184b2382507190a6c21e23 2023-02-11 03:53:50,371 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=37, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/aa18866afc8385f0b9c9afab2244ee1e/.tmp/family1/7d5ec00c5a4d4fc2b419f09b0a34abdf 2023-02-11 03:53:50,379 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/aa18866afc8385f0b9c9afab2244ee1e/.tmp/family1/7d5ec00c5a4d4fc2b419f09b0a34abdf as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/aa18866afc8385f0b9c9afab2244ee1e/family1/7d5ec00c5a4d4fc2b419f09b0a34abdf 2023-02-11 03:53:50,386 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/aa18866afc8385f0b9c9afab2244ee1e/family1/7d5ec00c5a4d4fc2b419f09b0a34abdf, entries=1, sequenceid=37, filesize=5.0 K 2023-02-11 03:53:50,388 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for aa18866afc8385f0b9c9afab2244ee1e in 845ms, sequenceid=37, compaction requested=true 2023-02-11 03:53:50,388 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for aa18866afc8385f0b9c9afab2244ee1e: 2023-02-11 03:53:50,388 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1676087622055.aa18866afc8385f0b9c9afab2244ee1e. 2023-02-11 03:53:50,388 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 03:53:50,388 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 03:53:50,388 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:53:50,388 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-02-11 03:53:50,388 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37011,1676087618428' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-02-11 03:53:50,391 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:53:50,391 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:50,391 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:53:50,391 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:53:50,391 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:53:50,391 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:50,391 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 03:53:50,392 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:53:50,392 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:53:50,392 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:50,392 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:53:50,393 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:53:50,393 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37011,1676087618428' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-02-11 03:53:50,393 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4db391ce[Count = 0] remaining members to acquire global barrier 2023-02-11 03:53:50,393 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-02-11 03:53:50,393 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:50,394 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:50,394 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:50,394 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:50,394 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-02-11 03:53:50,394 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:53:50,394 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-02-11 03:53:50,394 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 03:53:50,394 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase4.apache.org,37011,1676087618428' in zk 2023-02-11 03:53:50,396 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-02-11 03:53:50,396 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:53:50,396 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:53:50,396 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:53:50,396 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:53:50,396 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:53:50,396 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-02-11 03:53:50,397 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:53:50,397 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:53:50,397 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:50,397 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:53:50,398 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:53:50,398 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:50,398 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:53:50,399 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase4.apache.org,37011,1676087618428': 2023-02-11 03:53:50,399 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37011,1676087618428' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-02-11 03:53:50,399 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-02-11 03:53:50,399 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 03:53:50,399 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 03:53:50,399 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:50,399 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 03:53:50,401 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:50,401 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:50,401 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:50,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:50,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:53:50,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:53:50,401 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:53:50,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:50,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:53:50,401 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:53:50,401 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:53:50,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:53:50,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:50,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:50,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:53:50,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:50,403 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:53:50,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:53:50,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:53:50,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:50,404 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:53:50,406 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:53:50,406 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:50,406 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:50,406 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:53:50,406 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:53:50,406 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:53:50,406 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly because node does not exist (not an error) 2023-02-11 03:53:50,406 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:50,407 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:50,406 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-02-11 03:53:50,407 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 03:53:50,406 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:53:50,406 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:53:50,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:53:50,407 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:50,407 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-02-11 03:53:50,407 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:53:50,407 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 03:53:50,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:53:50,508 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 03:53:50,509 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 03:53:50,516 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 03:53:50,518 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-02-11 03:53:50,518 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:50,518 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-02-11 03:53:50,518 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:53:50,518 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-02-11 03:53:50,518 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-02-11 03:53:50,519 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:50,519 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:50,520 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:53:50,520 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:53:50,520 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:53:50,520 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:53:50,520 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:53:50,521 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 03:53:50,521 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:50,521 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:50,521 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 03:53:50,521 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:50,521 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:50,522 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-02-11 03:53:50,522 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:50,522 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-02-11 03:53:50,522 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:53:50,523 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-02-11 03:53:50,523 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 03:53:50,523 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 03:53:50,523 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1676087622055.aa18866afc8385f0b9c9afab2244ee1e. 2023-02-11 03:53:50,523 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1676087622055.aa18866afc8385f0b9c9afab2244ee1e. started... 2023-02-11 03:53:50,523 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing aa18866afc8385f0b9c9afab2244ee1e 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-11 03:53:50,942 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201511203142be550bb045a38d1495fe1ed9ae96 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201511203142be550bb045a38d1495fe1ed9ae96 2023-02-11 03:53:51,368 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=41, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/aa18866afc8385f0b9c9afab2244ee1e/.tmp/family1/8f8a3d45b6ea4dec8b8b95dc2ff44140 2023-02-11 03:53:51,376 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/aa18866afc8385f0b9c9afab2244ee1e/.tmp/family1/8f8a3d45b6ea4dec8b8b95dc2ff44140 as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/aa18866afc8385f0b9c9afab2244ee1e/family1/8f8a3d45b6ea4dec8b8b95dc2ff44140 2023-02-11 03:53:51,383 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/aa18866afc8385f0b9c9afab2244ee1e/family1/8f8a3d45b6ea4dec8b8b95dc2ff44140, entries=1, sequenceid=41, filesize=5.0 K 2023-02-11 03:53:51,384 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for aa18866afc8385f0b9c9afab2244ee1e in 861ms, sequenceid=41, compaction requested=true 2023-02-11 03:53:51,384 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for aa18866afc8385f0b9c9afab2244ee1e: 2023-02-11 03:53:51,384 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1676087622055.aa18866afc8385f0b9c9afab2244ee1e. 2023-02-11 03:53:51,384 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 03:53:51,385 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 03:53:51,385 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:53:51,385 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-02-11 03:53:51,385 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37011,1676087618428' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-02-11 03:53:51,387 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:53:51,387 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:51,387 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:53:51,387 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:53:51,387 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:53:51,387 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:51,387 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 03:53:51,387 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:53:51,387 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:53:51,388 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:51,388 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:53:51,388 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:53:51,389 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37011,1676087618428' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-02-11 03:53:51,389 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@c9b530c[Count = 0] remaining members to acquire global barrier 2023-02-11 03:53:51,389 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-02-11 03:53:51,389 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:51,390 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:51,390 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:51,390 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:51,390 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-02-11 03:53:51,390 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-02-11 03:53:51,390 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase4.apache.org,37011,1676087618428' in zk 2023-02-11 03:53:51,390 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:53:51,390 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 03:53:51,395 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:53:51,395 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-02-11 03:53:51,395 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:53:51,395 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:53:51,395 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:53:51,395 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-02-11 03:53:51,396 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:53:51,397 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:53:51,397 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:53:51,397 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:51,398 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:53:51,398 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:53:51,398 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:51,399 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:53:51,399 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase4.apache.org,37011,1676087618428': 2023-02-11 03:53:51,399 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37011,1676087618428' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-02-11 03:53:51,399 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-02-11 03:53:51,399 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 03:53:51,399 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 03:53:51,399 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:51,399 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 03:53:51,401 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:51,401 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:51,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:51,401 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:51,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:51,401 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:53:51,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:53:51,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:53:51,402 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:53:51,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:53:51,402 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:53:51,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:53:51,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:51,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:51,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:53:51,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:51,403 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:53:51,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:53:51,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:53:51,404 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:51,404 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:53:51,405 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:53:51,406 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:53:51,406 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:53:51,406 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:53:51,406 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:53:51,406 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-02-11 03:53:51,406 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 03:53:51,406 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:51,406 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:53:51,406 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly because node does not exist (not an error) 2023-02-11 03:53:51,406 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:51,406 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:53:51,406 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:51,406 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:51,406 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:51,406 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:53:51,406 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:53:51,406 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-02-11 03:53:51,407 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 03:53:51,507 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 03:53:51,508 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 03:53:51,513 DEBUG [Listener at localhost/46205] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-02-11 03:53:51,516 INFO [RS-EventLoopGroup-1-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:34744, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-02-11 03:53:51,517 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=39921] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}] of the table testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:51,529 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676087621935] procedure2.ProcedureExecutor(1029): Stored pid=12, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE 2023-02-11 03:53:51,529 DEBUG [PEWorker-4] locking.LockProcedure(309): LOCKED pid=12, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE 2023-02-11 03:53:51,531 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676087621935] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-11 03:53:51,531 INFO [PEWorker-4] procedure2.TimeoutExecutorThread(81): ADDED pid=12, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1676088231531 2023-02-11 03:53:51,532 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676087621935] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-02-11 03:53:51,537 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676087621935] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 10 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyFollowedByMonthly' and column 'family1' 2023-02-11 03:53:51,537 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676087621935] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-02-11 03:53:51,538 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676087621935] zookeeper.ReadOnlyZKClient(139): Connect 0x68fb9147 to 127.0.0.1:53383 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-11 03:53:51,542 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676087621935] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@256a6ddf, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-11 03:53:51,543 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-02-11 03:53:52,416 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201601032fd3195dee594aa68eae213da3686d8e to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201601032fd3195dee594aa68eae213da3686d8e 2023-02-11 03:53:52,417 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-02-11 03:53:52,436 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-11 03:53:52,438 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:45866, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-11 03:53:52,449 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-02-11 03:53:52,451 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:34754, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-02-11 03:53:52,485 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151228/family1/385f74bcbee2465c86d217152141b0b3 first=Optional[r3] last=Optional[r6] 2023-02-11 03:53:52,497 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthly,,1676087622055.aa18866afc8385f0b9c9afab2244ee1e., hostname=jenkins-hbase4.apache.org,37011,1676087618428, seqNum=2 for row with hfile group [{family1,hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151228/family1/385f74bcbee2465c86d217152141b0b3}] 2023-02-11 03:53:52,513 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.HStore(763): Validating hfile at hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151228/family1/385f74bcbee2465c86d217152141b0b3 for inclusion in aa18866afc8385f0b9c9afab2244ee1e/family1 2023-02-11 03:53:52,520 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.HStore(775): HFile bounds: first=r3 last=r6 2023-02-11 03:53:52,520 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.HStore(777): Region bounds: first= last= 2023-02-11 03:53:52,520 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.HRegion(2445): Flush status journal for aa18866afc8385f0b9c9afab2244ee1e: 2023-02-11 03:53:52,521 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151228/family1/385f74bcbee2465c86d217152141b0b3 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__vq2o1kef8a5nokhl0va10dmgblthekrtddak1apgk93i16klisr1gtb76eil84qj/family1/385f74bcbee2465c86d217152141b0b3 2023-02-11 03:53:52,525 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__vq2o1kef8a5nokhl0va10dmgblthekrtddak1apgk93i16klisr1gtb76eil84qj/family1/385f74bcbee2465c86d217152141b0b3 as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/aa18866afc8385f0b9c9afab2244ee1e/family1/2fb3c4bb6ad149599f14efea25aad864_SeqId_44_ 2023-02-11 03:53:52,526 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.HStore(849): Loaded HFile hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__vq2o1kef8a5nokhl0va10dmgblthekrtddak1apgk93i16klisr1gtb76eil84qj/family1/385f74bcbee2465c86d217152141b0b3 into aa18866afc8385f0b9c9afab2244ee1e/family1 as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/aa18866afc8385f0b9c9afab2244ee1e/family1/2fb3c4bb6ad149599f14efea25aad864_SeqId_44_ - updating store file list. 2023-02-11 03:53:52,531 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.HStore(882): Loaded HFile hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/aa18866afc8385f0b9c9afab2244ee1e/family1/2fb3c4bb6ad149599f14efea25aad864_SeqId_44_ into aa18866afc8385f0b9c9afab2244ee1e/family1 2023-02-11 03:53:52,532 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.HStore(855): Successfully loaded hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__vq2o1kef8a5nokhl0va10dmgblthekrtddak1apgk93i16klisr1gtb76eil84qj/family1/385f74bcbee2465c86d217152141b0b3 into aa18866afc8385f0b9c9afab2244ee1e/family1 (new location: hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/aa18866afc8385f0b9c9afab2244ee1e/family1/2fb3c4bb6ad149599f14efea25aad864_SeqId_44_) 2023-02-11 03:53:52,532 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__vq2o1kef8a5nokhl0va10dmgblthekrtddak1apgk93i16klisr1gtb76eil84qj/family1/385f74bcbee2465c86d217152141b0b3 2023-02-11 03:53:52,534 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] compactions.SortedCompactionPolicy(75): Selecting compaction from 11 store files, 0 compacting, 11 eligible, 16 blocking 2023-02-11 03:53:52,540 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 10 files of size 51357 starting at candidate #1 after considering 44 permutations with 44 in ratio 2023-02-11 03:53:52,542 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.HStore(1912): aa18866afc8385f0b9c9afab2244ee1e/family1 is initiating minor compaction 2023-02-11 03:53:52,543 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@2f486ff2; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-11 03:53:52,543 INFO [RS:0;jenkins-hbase4:37011-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of aa18866afc8385f0b9c9afab2244ee1e/family1 in testMajorCompactionWithWeeklyFollowedByMonthly,,1676087622055.aa18866afc8385f0b9c9afab2244ee1e. 2023-02-11 03:53:52,544 INFO [RS:0;jenkins-hbase4:37011-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/aa18866afc8385f0b9c9afab2244ee1e/family1/2137b3c52d8641c39cd08ea5024bbed9, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/aa18866afc8385f0b9c9afab2244ee1e/family1/68470dbad13e4b159f7094730288d5b8, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/aa18866afc8385f0b9c9afab2244ee1e/family1/ba8f9ade0aeb4025827d6678e12285fa, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/aa18866afc8385f0b9c9afab2244ee1e/family1/9a8fd510cb944335a1130636196e0b6f, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/aa18866afc8385f0b9c9afab2244ee1e/family1/8b1f001b1b1343cea954814d01294a86, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/aa18866afc8385f0b9c9afab2244ee1e/family1/070b081fe1c64ad081eea06b4e2f0f44, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/aa18866afc8385f0b9c9afab2244ee1e/family1/7f8e9ad2e58346d6af31ab4cea7e13e7, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/aa18866afc8385f0b9c9afab2244ee1e/family1/7d5ec00c5a4d4fc2b419f09b0a34abdf, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/aa18866afc8385f0b9c9afab2244ee1e/family1/8f8a3d45b6ea4dec8b8b95dc2ff44140, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/aa18866afc8385f0b9c9afab2244ee1e/family1/2fb3c4bb6ad149599f14efea25aad864_SeqId_44_] into tmpdir=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/aa18866afc8385f0b9c9afab2244ee1e/.tmp, totalSize=50.2 K 2023-02-11 03:53:52,544 DEBUG [RS:0;jenkins-hbase4:37011-shortCompactions-0] compactions.Compactor(207): Compacting 2137b3c52d8641c39cd08ea5024bbed9, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=9 2023-02-11 03:53:52,545 DEBUG [RS:0;jenkins-hbase4:37011-shortCompactions-0] compactions.Compactor(207): Compacting 68470dbad13e4b159f7094730288d5b8, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=13 2023-02-11 03:53:52,545 DEBUG [RS:0;jenkins-hbase4:37011-shortCompactions-0] compactions.Compactor(207): Compacting ba8f9ade0aeb4025827d6678e12285fa, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=17 2023-02-11 03:53:52,545 DEBUG [RS:0;jenkins-hbase4:37011-shortCompactions-0] compactions.Compactor(207): Compacting 9a8fd510cb944335a1130636196e0b6f, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=21 2023-02-11 03:53:52,545 DEBUG [RS:0;jenkins-hbase4:37011-shortCompactions-0] compactions.Compactor(207): Compacting 8b1f001b1b1343cea954814d01294a86, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=25 2023-02-11 03:53:52,545 DEBUG [RS:0;jenkins-hbase4:37011-shortCompactions-0] compactions.Compactor(207): Compacting 070b081fe1c64ad081eea06b4e2f0f44, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=29 2023-02-11 03:53:52,545 DEBUG [RS:0;jenkins-hbase4:37011-shortCompactions-0] compactions.Compactor(207): Compacting 7f8e9ad2e58346d6af31ab4cea7e13e7, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33 2023-02-11 03:53:52,545 DEBUG [RS:0;jenkins-hbase4:37011-shortCompactions-0] compactions.Compactor(207): Compacting 7d5ec00c5a4d4fc2b419f09b0a34abdf, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=37 2023-02-11 03:53:52,545 DEBUG [RS:0;jenkins-hbase4:37011-shortCompactions-0] compactions.Compactor(207): Compacting 8f8a3d45b6ea4dec8b8b95dc2ff44140, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=41 2023-02-11 03:53:52,545 DEBUG [RS:0;jenkins-hbase4:37011-shortCompactions-0] compactions.Compactor(207): Compacting 2fb3c4bb6ad149599f14efea25aad864_SeqId_44_, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=45 2023-02-11 03:53:52,557 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-02-11 03:53:52,559 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-11 03:53:52,565 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201512286bfb9c6fdf6b46c6bedbcaaded338cdc to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201512286bfb9c6fdf6b46c6bedbcaaded338cdc 2023-02-11 03:53:52,566 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151231ed2ec05b3b78468f98c65b65ee206b78 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151231ed2ec05b3b78468f98c65b65ee206b78 2023-02-11 03:53:52,569 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201601012f7ce5aa547e409bbfb59cf496dc0285 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201601012f7ce5aa547e409bbfb59cf496dc0285 2023-02-11 03:53:52,570 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e2016010378ad5eaaf02d46b4b6bd5bf99a092408 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e2016010378ad5eaaf02d46b4b6bd5bf99a092408 2023-02-11 03:53:52,570 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthly 4 expired mob files are deleted 2023-02-11 03:53:52,570 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151228' 2023-02-11 03:53:52,571 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-02-11 03:53:52,571 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-02-11 03:53:52,571 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-02-11 03:53:52,571 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-02-11 03:53:52,571 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-02-11 03:53:52,571 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-02-11 03:53:52,572 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-02-11 03:53:52,572 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-02-11 03:53:52,601 INFO [RS:0;jenkins-hbase4:37011-shortCompactions-0] throttle.PressureAwareThroughputController(145): aa18866afc8385f0b9c9afab2244ee1e#family1#compaction#10 average throughput is 0.13 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-11 03:53:53,027 DEBUG [RS:0;jenkins-hbase4:37011-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/aa18866afc8385f0b9c9afab2244ee1e/.tmp/family1/8abb54c27fa046c7b3b6afaa6ced22d2 as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/aa18866afc8385f0b9c9afab2244ee1e/family1/8abb54c27fa046c7b3b6afaa6ced22d2 2023-02-11 03:53:53,043 INFO [RS:0;jenkins-hbase4:37011-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 10 file(s) in aa18866afc8385f0b9c9afab2244ee1e/family1 of aa18866afc8385f0b9c9afab2244ee1e into 8abb54c27fa046c7b3b6afaa6ced22d2(size=6.6 K), total size for store is 182.5 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-11 03:53:53,043 DEBUG [RS:0;jenkins-hbase4:37011-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for aa18866afc8385f0b9c9afab2244ee1e: 2023-02-11 03:53:53,045 INFO [RS:0;jenkins-hbase4:37011-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithWeeklyFollowedByMonthly,,1676087622055.aa18866afc8385f0b9c9afab2244ee1e., storeName=family1, fileCount=10, fileSize=50.2 K (5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.5 K), priority=2, time=1676087632541; duration=0sec 2023-02-11 03:53:53,046 DEBUG [RS:0;jenkins-hbase4:37011-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-11 03:53:53,096 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/aa18866afc8385f0b9c9afab2244ee1e/family1/2137b3c52d8641c39cd08ea5024bbed9, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/aa18866afc8385f0b9c9afab2244ee1e/family1/68470dbad13e4b159f7094730288d5b8, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/aa18866afc8385f0b9c9afab2244ee1e/family1/ba8f9ade0aeb4025827d6678e12285fa, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/aa18866afc8385f0b9c9afab2244ee1e/family1/9a8fd510cb944335a1130636196e0b6f, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/aa18866afc8385f0b9c9afab2244ee1e/family1/8b1f001b1b1343cea954814d01294a86, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/aa18866afc8385f0b9c9afab2244ee1e/family1/070b081fe1c64ad081eea06b4e2f0f44, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/aa18866afc8385f0b9c9afab2244ee1e/family1/7f8e9ad2e58346d6af31ab4cea7e13e7, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/aa18866afc8385f0b9c9afab2244ee1e/family1/7d5ec00c5a4d4fc2b419f09b0a34abdf, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/aa18866afc8385f0b9c9afab2244ee1e/family1/8f8a3d45b6ea4dec8b8b95dc2ff44140, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/aa18866afc8385f0b9c9afab2244ee1e/family1/2fb3c4bb6ad149599f14efea25aad864_SeqId_44_] to archive 2023-02-11 03:53:53,097 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-11 03:53:53,100 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/aa18866afc8385f0b9c9afab2244ee1e/family1/2137b3c52d8641c39cd08ea5024bbed9 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/aa18866afc8385f0b9c9afab2244ee1e/family1/2137b3c52d8641c39cd08ea5024bbed9 2023-02-11 03:53:53,102 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/aa18866afc8385f0b9c9afab2244ee1e/family1/68470dbad13e4b159f7094730288d5b8 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/aa18866afc8385f0b9c9afab2244ee1e/family1/68470dbad13e4b159f7094730288d5b8 2023-02-11 03:53:53,103 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/aa18866afc8385f0b9c9afab2244ee1e/family1/ba8f9ade0aeb4025827d6678e12285fa to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/aa18866afc8385f0b9c9afab2244ee1e/family1/ba8f9ade0aeb4025827d6678e12285fa 2023-02-11 03:53:53,105 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/aa18866afc8385f0b9c9afab2244ee1e/family1/9a8fd510cb944335a1130636196e0b6f to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/aa18866afc8385f0b9c9afab2244ee1e/family1/9a8fd510cb944335a1130636196e0b6f 2023-02-11 03:53:53,106 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/aa18866afc8385f0b9c9afab2244ee1e/family1/8b1f001b1b1343cea954814d01294a86 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/aa18866afc8385f0b9c9afab2244ee1e/family1/8b1f001b1b1343cea954814d01294a86 2023-02-11 03:53:53,108 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/aa18866afc8385f0b9c9afab2244ee1e/family1/070b081fe1c64ad081eea06b4e2f0f44 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/aa18866afc8385f0b9c9afab2244ee1e/family1/070b081fe1c64ad081eea06b4e2f0f44 2023-02-11 03:53:53,109 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/aa18866afc8385f0b9c9afab2244ee1e/family1/7f8e9ad2e58346d6af31ab4cea7e13e7 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/aa18866afc8385f0b9c9afab2244ee1e/family1/7f8e9ad2e58346d6af31ab4cea7e13e7 2023-02-11 03:53:53,110 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/aa18866afc8385f0b9c9afab2244ee1e/family1/7d5ec00c5a4d4fc2b419f09b0a34abdf to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/aa18866afc8385f0b9c9afab2244ee1e/family1/7d5ec00c5a4d4fc2b419f09b0a34abdf 2023-02-11 03:53:53,112 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/aa18866afc8385f0b9c9afab2244ee1e/family1/8f8a3d45b6ea4dec8b8b95dc2ff44140 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/aa18866afc8385f0b9c9afab2244ee1e/family1/8f8a3d45b6ea4dec8b8b95dc2ff44140 2023-02-11 03:53:53,114 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/aa18866afc8385f0b9c9afab2244ee1e/family1/2fb3c4bb6ad149599f14efea25aad864_SeqId_44_ to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/aa18866afc8385f0b9c9afab2244ee1e/family1/2fb3c4bb6ad149599f14efea25aad864_SeqId_44_ 2023-02-11 03:53:53,472 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151205a14bc3d93289469f8d847ec35519d789 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151205a14bc3d93289469f8d847ec35519d789 2023-02-11 03:53:53,474 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-02-11 03:53:53,503 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151130/family1/36404044920846e8ad377190e3271d87 first=Optional[r00] last=Optional[r2] 2023-02-11 03:53:53,504 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthly,,1676087622055.aa18866afc8385f0b9c9afab2244ee1e., hostname=jenkins-hbase4.apache.org,37011,1676087618428, seqNum=2 for row with hfile group [{family1,hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151130/family1/36404044920846e8ad377190e3271d87}] 2023-02-11 03:53:53,506 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.HStore(763): Validating hfile at hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151130/family1/36404044920846e8ad377190e3271d87 for inclusion in aa18866afc8385f0b9c9afab2244ee1e/family1 2023-02-11 03:53:53,511 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.HStore(775): HFile bounds: first=r00 last=r2 2023-02-11 03:53:53,511 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.HStore(777): Region bounds: first= last= 2023-02-11 03:53:53,512 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.HRegion(2445): Flush status journal for aa18866afc8385f0b9c9afab2244ee1e: 2023-02-11 03:53:53,513 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151130/family1/36404044920846e8ad377190e3271d87 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__r3dq6cf432842cdmsgaq3lnia8ulaatmu6ittq5coal72ng43d76talmmspjqi2r/family1/36404044920846e8ad377190e3271d87 2023-02-11 03:53:53,515 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__r3dq6cf432842cdmsgaq3lnia8ulaatmu6ittq5coal72ng43d76talmmspjqi2r/family1/36404044920846e8ad377190e3271d87 as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/aa18866afc8385f0b9c9afab2244ee1e/family1/98ca33b3834147bdaee10f4cbccce0f0_SeqId_47_ 2023-02-11 03:53:53,516 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.HStore(849): Loaded HFile hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__r3dq6cf432842cdmsgaq3lnia8ulaatmu6ittq5coal72ng43d76talmmspjqi2r/family1/36404044920846e8ad377190e3271d87 into aa18866afc8385f0b9c9afab2244ee1e/family1 as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/aa18866afc8385f0b9c9afab2244ee1e/family1/98ca33b3834147bdaee10f4cbccce0f0_SeqId_47_ - updating store file list. 2023-02-11 03:53:53,523 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.HStore(882): Loaded HFile hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/aa18866afc8385f0b9c9afab2244ee1e/family1/98ca33b3834147bdaee10f4cbccce0f0_SeqId_47_ into aa18866afc8385f0b9c9afab2244ee1e/family1 2023-02-11 03:53:53,523 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.HStore(855): Successfully loaded hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__r3dq6cf432842cdmsgaq3lnia8ulaatmu6ittq5coal72ng43d76talmmspjqi2r/family1/36404044920846e8ad377190e3271d87 into aa18866afc8385f0b9c9afab2244ee1e/family1 (new location: hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/aa18866afc8385f0b9c9afab2244ee1e/family1/98ca33b3834147bdaee10f4cbccce0f0_SeqId_47_) 2023-02-11 03:53:53,524 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__r3dq6cf432842cdmsgaq3lnia8ulaatmu6ittq5coal72ng43d76talmmspjqi2r/family1/36404044920846e8ad377190e3271d87 2023-02-11 03:53:53,524 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-02-11 03:53:53,525 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 367249 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-02-11 03:53:53,525 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.HStore(1912): aa18866afc8385f0b9c9afab2244ee1e/family1 is initiating minor compaction (all files) 2023-02-11 03:53:53,525 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@34893a6c; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-11 03:53:53,525 INFO [RS:0;jenkins-hbase4:37011-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of aa18866afc8385f0b9c9afab2244ee1e/family1 in testMajorCompactionWithWeeklyFollowedByMonthly,,1676087622055.aa18866afc8385f0b9c9afab2244ee1e. 2023-02-11 03:53:53,527 INFO [RS:0;jenkins-hbase4:37011-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/aa18866afc8385f0b9c9afab2244ee1e/family1/7ce3a1bf235e4eb29c256807a397a523, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/aa18866afc8385f0b9c9afab2244ee1e/family1/8abb54c27fa046c7b3b6afaa6ced22d2, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/aa18866afc8385f0b9c9afab2244ee1e/family1/98ca33b3834147bdaee10f4cbccce0f0_SeqId_47_] into tmpdir=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/aa18866afc8385f0b9c9afab2244ee1e/.tmp, totalSize=358.6 K 2023-02-11 03:53:53,527 DEBUG [RS:0;jenkins-hbase4:37011-shortCompactions-0] compactions.Compactor(207): Compacting 7ce3a1bf235e4eb29c256807a397a523, keycount=1000, bloomtype=ROW, size=175.8 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1448878842014 2023-02-11 03:53:53,528 DEBUG [RS:0;jenkins-hbase4:37011-shortCompactions-0] compactions.Compactor(207): Compacting 8abb54c27fa046c7b3b6afaa6ced22d2, keycount=9, bloomtype=ROW, size=6.6 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1441621242014 2023-02-11 03:53:53,528 DEBUG [RS:0;jenkins-hbase4:37011-shortCompactions-0] compactions.Compactor(207): Compacting 98ca33b3834147bdaee10f4cbccce0f0_SeqId_47_, keycount=1002, bloomtype=ROW, size=176.2 K, encoding=NONE, compression=NONE, seqNum=48, earliestPutTs=1448878842014 2023-02-11 03:53:53,529 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-02-11 03:53:53,530 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-11 03:53:53,532 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151130da6aae65587c4bafb955b834ccd75afc to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151130da6aae65587c4bafb955b834ccd75afc 2023-02-11 03:53:53,534 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201512016848f71c04514a18b9891bcd2b51ffe6 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201512016848f71c04514a18b9891bcd2b51ffe6 2023-02-11 03:53:53,536 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151205895aab49be144fd6b4e1f5112c1ddc09 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151205895aab49be144fd6b4e1f5112c1ddc09 2023-02-11 03:53:53,536 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthly 3 expired mob files are deleted 2023-02-11 03:53:53,536 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151130' 2023-02-11 03:53:53,536 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 3 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-02-11 03:53:53,537 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676087621935] compactions.PartitionedMobCompactor(354): After compaction, there are 5 mob files. table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-02-11 03:53:53,537 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676087621935] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-11 03:53:53,538 DEBUG [PEWorker-5] locking.LockProcedure(242): UNLOCKED pid=12, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE 2023-02-11 03:53:53,540 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=12, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE in 2.0170 sec 2023-02-11 03:53:53,564 INFO [RS:0;jenkins-hbase4:37011-shortCompactions-0] throttle.PressureAwareThroughputController(145): aa18866afc8385f0b9c9afab2244ee1e#family1#compaction#11 average throughput is 7.51 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-11 03:53:53,696 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] master.HMaster$12(2441): Client=jenkins//172.31.14.131 modify table testMajorCompactionWithWeeklyFollowedByMonthly from 'testMajorCompactionWithWeeklyFollowedByMonthly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} to 'testMajorCompactionWithWeeklyFollowedByMonthly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'} 2023-02-11 03:53:53,707 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] procedure2.ProcedureExecutor(1029): Stored pid=13, state=RUNNABLE:MODIFY_TABLE_PREPARE; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:54,124 DEBUG [PEWorker-2] util.FSTableDescriptors(570): Wrote into hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/.tabledesc/.tableinfo.0000000002 2023-02-11 03:53:54,125 DEBUG [PEWorker-2] util.FSTableDescriptors(519): Deleted hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/.tabledesc/.tableinfo.0000000001 2023-02-11 03:53:54,125 INFO [PEWorker-2] util.FSTableDescriptors(293): Updated tableinfo=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/.tabledesc/.tableinfo.0000000002 2023-02-11 03:53:54,130 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=14, ppid=13, state=RUNNABLE:REOPEN_TABLE_REGIONS_GET_REGIONS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly}] 2023-02-11 03:53:54,143 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=15, ppid=14, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly, region=aa18866afc8385f0b9c9afab2244ee1e, REOPEN/MOVE}] 2023-02-11 03:53:54,144 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=15, ppid=14, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly, region=aa18866afc8385f0b9c9afab2244ee1e, REOPEN/MOVE 2023-02-11 03:53:54,145 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=15 updating hbase:meta row=aa18866afc8385f0b9c9afab2244ee1e, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:53:54,145 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthly,,1676087622055.aa18866afc8385f0b9c9afab2244ee1e.","families":{"info":[{"qualifier":"regioninfo","vlen":80,"tag":[],"timestamp":"1676087634145"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676087634145"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676087634145"}]},"ts":"1676087634145"} 2023-02-11 03:53:54,148 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=16, ppid=15, state=RUNNABLE; CloseRegionProcedure aa18866afc8385f0b9c9afab2244ee1e, server=jenkins-hbase4.apache.org,37011,1676087618428}] 2023-02-11 03:53:54,307 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close aa18866afc8385f0b9c9afab2244ee1e 2023-02-11 03:53:54,307 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing aa18866afc8385f0b9c9afab2244ee1e, disabling compactions & flushes 2023-02-11 03:53:54,308 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1857): waiting for 1 compactions to complete for region testMajorCompactionWithWeeklyFollowedByMonthly,,1676087622055.aa18866afc8385f0b9c9afab2244ee1e. 2023-02-11 03:53:54,803 DEBUG [RS:0;jenkins-hbase4:37011-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/aa18866afc8385f0b9c9afab2244ee1e/.tmp/family1/89c2cbd0ec104dafb16ba26f3c932288 as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/aa18866afc8385f0b9c9afab2244ee1e/family1/89c2cbd0ec104dafb16ba26f3c932288 2023-02-11 03:53:54,810 INFO [RS:0;jenkins-hbase4:37011-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in aa18866afc8385f0b9c9afab2244ee1e/family1 of aa18866afc8385f0b9c9afab2244ee1e into 89c2cbd0ec104dafb16ba26f3c932288(size=177.2 K), total size for store is 177.2 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-02-11 03:53:54,810 DEBUG [RS:0;jenkins-hbase4:37011-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for aa18866afc8385f0b9c9afab2244ee1e: 2023-02-11 03:53:54,810 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testMajorCompactionWithWeeklyFollowedByMonthly,,1676087622055.aa18866afc8385f0b9c9afab2244ee1e. 2023-02-11 03:53:54,810 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthly,,1676087622055.aa18866afc8385f0b9c9afab2244ee1e. 2023-02-11 03:53:54,810 INFO [RS:0;jenkins-hbase4:37011-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithWeeklyFollowedByMonthly,,1676087622055.aa18866afc8385f0b9c9afab2244ee1e., storeName=family1, fileCount=3, fileSize=358.6 K (175.8 K, 6.6 K, 176.2 K), priority=2, time=1676087633525; duration=1sec 2023-02-11 03:53:54,810 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthly,,1676087622055.aa18866afc8385f0b9c9afab2244ee1e. after waiting 0 ms 2023-02-11 03:53:54,810 DEBUG [RS:0;jenkins-hbase4:37011-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-11 03:53:54,810 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthly,,1676087622055.aa18866afc8385f0b9c9afab2244ee1e. 2023-02-11 03:53:54,813 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthly,,1676087622055.aa18866afc8385f0b9c9afab2244ee1e.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/aa18866afc8385f0b9c9afab2244ee1e/family1/7ce3a1bf235e4eb29c256807a397a523, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/aa18866afc8385f0b9c9afab2244ee1e/family1/8abb54c27fa046c7b3b6afaa6ced22d2, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/aa18866afc8385f0b9c9afab2244ee1e/family1/98ca33b3834147bdaee10f4cbccce0f0_SeqId_47_] to archive 2023-02-11 03:53:54,814 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthly,,1676087622055.aa18866afc8385f0b9c9afab2244ee1e.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-11 03:53:54,815 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthly,,1676087622055.aa18866afc8385f0b9c9afab2244ee1e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/aa18866afc8385f0b9c9afab2244ee1e/family1/7ce3a1bf235e4eb29c256807a397a523 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/aa18866afc8385f0b9c9afab2244ee1e/family1/7ce3a1bf235e4eb29c256807a397a523 2023-02-11 03:53:54,817 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthly,,1676087622055.aa18866afc8385f0b9c9afab2244ee1e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/aa18866afc8385f0b9c9afab2244ee1e/family1/8abb54c27fa046c7b3b6afaa6ced22d2 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/aa18866afc8385f0b9c9afab2244ee1e/family1/8abb54c27fa046c7b3b6afaa6ced22d2 2023-02-11 03:53:54,818 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthly,,1676087622055.aa18866afc8385f0b9c9afab2244ee1e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/aa18866afc8385f0b9c9afab2244ee1e/family1/98ca33b3834147bdaee10f4cbccce0f0_SeqId_47_ to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/aa18866afc8385f0b9c9afab2244ee1e/family1/98ca33b3834147bdaee10f4cbccce0f0_SeqId_47_ 2023-02-11 03:53:54,824 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/aa18866afc8385f0b9c9afab2244ee1e/recovered.edits/50.seqid, newMaxSeqId=50, maxSeqId=1 2023-02-11 03:53:54,825 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testMajorCompactionWithWeeklyFollowedByMonthly,,1676087622055.aa18866afc8385f0b9c9afab2244ee1e. 2023-02-11 03:53:54,825 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for aa18866afc8385f0b9c9afab2244ee1e: Waiting for close lock at 1676087634307Running coprocessor pre-close hooks at 1676087634307Disabling compacts and flushes for region at 1676087634307Disabling writes for close at 1676087634810 (+503 ms)Writing region close event to WAL at 1676087634821 (+11 ms)Running coprocessor post-close hooks at 1676087634824 (+3 ms)Closed at 1676087634825 (+1 ms) 2023-02-11 03:53:54,825 WARN [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(3509): Not adding moved region record: aa18866afc8385f0b9c9afab2244ee1e to self. 2023-02-11 03:53:54,827 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed aa18866afc8385f0b9c9afab2244ee1e 2023-02-11 03:53:54,827 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=15 updating hbase:meta row=aa18866afc8385f0b9c9afab2244ee1e, regionState=CLOSED 2023-02-11 03:53:54,828 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyFollowedByMonthly,,1676087622055.aa18866afc8385f0b9c9afab2244ee1e.","families":{"info":[{"qualifier":"regioninfo","vlen":80,"tag":[],"timestamp":"1676087634827"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676087634827"}]},"ts":"1676087634827"} 2023-02-11 03:53:54,831 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=16, resume processing ppid=15 2023-02-11 03:53:54,831 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=16, ppid=15, state=SUCCESS; CloseRegionProcedure aa18866afc8385f0b9c9afab2244ee1e, server=jenkins-hbase4.apache.org,37011,1676087618428 in 681 msec 2023-02-11 03:53:54,832 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=15, ppid=14, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly, region=aa18866afc8385f0b9c9afab2244ee1e, REOPEN/MOVE; state=CLOSED, location=jenkins-hbase4.apache.org,37011,1676087618428; forceNewPlan=false, retain=true 2023-02-11 03:53:54,983 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=15 updating hbase:meta row=aa18866afc8385f0b9c9afab2244ee1e, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:53:54,983 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthly,,1676087622055.aa18866afc8385f0b9c9afab2244ee1e.","families":{"info":[{"qualifier":"regioninfo","vlen":80,"tag":[],"timestamp":"1676087634983"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676087634983"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676087634983"}]},"ts":"1676087634983"} 2023-02-11 03:53:54,985 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=17, ppid=15, state=RUNNABLE; OpenRegionProcedure aa18866afc8385f0b9c9afab2244ee1e, server=jenkins-hbase4.apache.org,37011,1676087618428}] 2023-02-11 03:53:55,142 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionWithWeeklyFollowedByMonthly,,1676087622055.aa18866afc8385f0b9c9afab2244ee1e. 2023-02-11 03:53:55,143 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => aa18866afc8385f0b9c9afab2244ee1e, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthly,,1676087622055.aa18866afc8385f0b9c9afab2244ee1e.', STARTKEY => '', ENDKEY => ''} 2023-02-11 03:53:55,143 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyFollowedByMonthly aa18866afc8385f0b9c9afab2244ee1e 2023-02-11 03:53:55,143 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testMajorCompactionWithWeeklyFollowedByMonthly,,1676087622055.aa18866afc8385f0b9c9afab2244ee1e.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-11 03:53:55,143 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for aa18866afc8385f0b9c9afab2244ee1e 2023-02-11 03:53:55,143 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for aa18866afc8385f0b9c9afab2244ee1e 2023-02-11 03:53:55,145 INFO [StoreOpener-aa18866afc8385f0b9c9afab2244ee1e-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region aa18866afc8385f0b9c9afab2244ee1e 2023-02-11 03:53:55,146 DEBUG [StoreOpener-aa18866afc8385f0b9c9afab2244ee1e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/aa18866afc8385f0b9c9afab2244ee1e/family1 2023-02-11 03:53:55,146 DEBUG [StoreOpener-aa18866afc8385f0b9c9afab2244ee1e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/aa18866afc8385f0b9c9afab2244ee1e/family1 2023-02-11 03:53:55,147 INFO [StoreOpener-aa18866afc8385f0b9c9afab2244ee1e-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region aa18866afc8385f0b9c9afab2244ee1e columnFamilyName family1 2023-02-11 03:53:55,154 DEBUG [StoreOpener-aa18866afc8385f0b9c9afab2244ee1e-1] regionserver.HStore(539): loaded hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/aa18866afc8385f0b9c9afab2244ee1e/family1/89c2cbd0ec104dafb16ba26f3c932288 2023-02-11 03:53:55,155 INFO [StoreOpener-aa18866afc8385f0b9c9afab2244ee1e-1] regionserver.HStore(310): Store=aa18866afc8385f0b9c9afab2244ee1e/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 03:53:55,156 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/aa18866afc8385f0b9c9afab2244ee1e 2023-02-11 03:53:55,157 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/aa18866afc8385f0b9c9afab2244ee1e 2023-02-11 03:53:55,160 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for aa18866afc8385f0b9c9afab2244ee1e 2023-02-11 03:53:55,161 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened aa18866afc8385f0b9c9afab2244ee1e; next sequenceid=51; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=59787295, jitterRate=-0.10909987986087799}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-11 03:53:55,161 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for aa18866afc8385f0b9c9afab2244ee1e: Running coprocessor pre-open hook at 1676087635143Writing region info on filesystem at 1676087635143Initializing all the Stores at 1676087635144 (+1 ms)Instantiating store for column family {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'} at 1676087635144Cleaning up temporary data from old regions at 1676087635158 (+14 ms)Cleaning up detritus from prior splits at 1676087635158Running coprocessor post-open hooks at 1676087635161 (+3 ms)Region opened successfully at 1676087635161 2023-02-11 03:53:55,162 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionWithWeeklyFollowedByMonthly,,1676087622055.aa18866afc8385f0b9c9afab2244ee1e., pid=17, masterSystemTime=1676087635138 2023-02-11 03:53:55,164 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionWithWeeklyFollowedByMonthly,,1676087622055.aa18866afc8385f0b9c9afab2244ee1e. 2023-02-11 03:53:55,164 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionWithWeeklyFollowedByMonthly,,1676087622055.aa18866afc8385f0b9c9afab2244ee1e. 2023-02-11 03:53:55,165 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=15 updating hbase:meta row=aa18866afc8385f0b9c9afab2244ee1e, regionState=OPEN, openSeqNum=51, regionLocation=jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:53:55,165 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyFollowedByMonthly,,1676087622055.aa18866afc8385f0b9c9afab2244ee1e.","families":{"info":[{"qualifier":"regioninfo","vlen":80,"tag":[],"timestamp":"1676087635165"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676087635165"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676087635165"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676087635165"}]},"ts":"1676087635165"} 2023-02-11 03:53:55,169 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=17, resume processing ppid=15 2023-02-11 03:53:55,169 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=17, ppid=15, state=SUCCESS; OpenRegionProcedure aa18866afc8385f0b9c9afab2244ee1e, server=jenkins-hbase4.apache.org,37011,1676087618428 in 182 msec 2023-02-11 03:53:55,171 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=15, resume processing ppid=14 2023-02-11 03:53:55,171 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=15, ppid=14, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly, region=aa18866afc8385f0b9c9afab2244ee1e, REOPEN/MOVE in 1.0260 sec 2023-02-11 03:53:55,174 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=14, resume processing ppid=13 2023-02-11 03:53:55,174 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=14, ppid=13, state=SUCCESS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly in 1.0430 sec 2023-02-11 03:53:55,176 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=13, state=SUCCESS; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly in 1.4750 sec 2023-02-11 03:53:55,179 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] master.MasterRpcServices(1227): Checking to see if procedure is done pid=13 2023-02-11 03:53:55,186 INFO [Listener at localhost/46205] compactions.TestMobCompactor(278): alter status finished 2023-02-11 03:53:55,187 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=39921] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'}] of the table testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-11 03:53:55,189 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676087621935] procedure2.ProcedureExecutor(1029): Stored pid=18, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE 2023-02-11 03:53:55,189 DEBUG [PEWorker-3] locking.LockProcedure(309): LOCKED pid=18, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE 2023-02-11 03:53:55,190 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676087621935] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-11 03:53:55,190 INFO [PEWorker-3] procedure2.TimeoutExecutorThread(81): ADDED pid=18, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1676088235190 2023-02-11 03:53:55,191 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676087621935] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-02-11 03:53:55,192 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676087621935] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 5 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyFollowedByMonthly' and column 'family1' 2023-02-11 03:53:55,192 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676087621935] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-02-11 03:53:55,193 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676087621935] zookeeper.ReadOnlyZKClient(139): Connect 0x7228ad49 to 127.0.0.1:53383 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-11 03:53:55,202 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676087621935] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@18316201, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-11 03:53:55,202 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-02-11 03:53:55,203 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-02-11 03:53:55,203 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-02-11 03:53:55,203 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-02-11 03:53:55,203 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-02-11 03:53:55,637 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151128ac608baa3d9f4cbe952a960707581d24 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151128ac608baa3d9f4cbe952a960707581d24 2023-02-11 03:53:55,638 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-02-11 03:53:55,651 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-11 03:53:55,652 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:60718, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-11 03:53:55,661 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-02-11 03:53:55,663 INFO [RS-EventLoopGroup-1-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:35300, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-02-11 03:53:55,675 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151101/family1/3155e7a1aac943b3b9608a3f9dc251e1 first=Optional[r05] last=Optional[r8] 2023-02-11 03:53:55,676 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthly,,1676087622055.aa18866afc8385f0b9c9afab2244ee1e., hostname=jenkins-hbase4.apache.org,37011,1676087618428, seqNum=51 for row with hfile group [{family1,hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151101/family1/3155e7a1aac943b3b9608a3f9dc251e1}] 2023-02-11 03:53:55,682 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.HStore(763): Validating hfile at hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151101/family1/3155e7a1aac943b3b9608a3f9dc251e1 for inclusion in aa18866afc8385f0b9c9afab2244ee1e/family1 2023-02-11 03:53:55,685 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.HStore(775): HFile bounds: first=r05 last=r8 2023-02-11 03:53:55,686 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.HStore(777): Region bounds: first= last= 2023-02-11 03:53:55,686 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.HRegion(2445): Flush status journal for aa18866afc8385f0b9c9afab2244ee1e: 2023-02-11 03:53:55,687 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151101/family1/3155e7a1aac943b3b9608a3f9dc251e1 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__ng0tgo89cmsfnsfp6l3r71i2o6ukup9sb8sqeggolj8lbvcfsm3ei7b3dvhkg1d7/family1/3155e7a1aac943b3b9608a3f9dc251e1 2023-02-11 03:53:55,691 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__ng0tgo89cmsfnsfp6l3r71i2o6ukup9sb8sqeggolj8lbvcfsm3ei7b3dvhkg1d7/family1/3155e7a1aac943b3b9608a3f9dc251e1 as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/aa18866afc8385f0b9c9afab2244ee1e/family1/dba93b7189f64f3ab5b7b7e0dcc6f234_SeqId_53_ 2023-02-11 03:53:55,691 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.HStore(849): Loaded HFile hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__ng0tgo89cmsfnsfp6l3r71i2o6ukup9sb8sqeggolj8lbvcfsm3ei7b3dvhkg1d7/family1/3155e7a1aac943b3b9608a3f9dc251e1 into aa18866afc8385f0b9c9afab2244ee1e/family1 as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/aa18866afc8385f0b9c9afab2244ee1e/family1/dba93b7189f64f3ab5b7b7e0dcc6f234_SeqId_53_ - updating store file list. 2023-02-11 03:53:55,697 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.HStore(882): Loaded HFile hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/aa18866afc8385f0b9c9afab2244ee1e/family1/dba93b7189f64f3ab5b7b7e0dcc6f234_SeqId_53_ into aa18866afc8385f0b9c9afab2244ee1e/family1 2023-02-11 03:53:55,697 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.HStore(855): Successfully loaded hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__ng0tgo89cmsfnsfp6l3r71i2o6ukup9sb8sqeggolj8lbvcfsm3ei7b3dvhkg1d7/family1/3155e7a1aac943b3b9608a3f9dc251e1 into aa18866afc8385f0b9c9afab2244ee1e/family1 (new location: hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/aa18866afc8385f0b9c9afab2244ee1e/family1/dba93b7189f64f3ab5b7b7e0dcc6f234_SeqId_53_) 2023-02-11 03:53:55,698 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__ng0tgo89cmsfnsfp6l3r71i2o6ukup9sb8sqeggolj8lbvcfsm3ei7b3dvhkg1d7/family1/3155e7a1aac943b3b9608a3f9dc251e1 2023-02-11 03:53:55,700 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-02-11 03:53:55,701 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-11 03:53:55,703 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201511203142be550bb045a38d1495fe1ed9ae96 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201511203142be550bb045a38d1495fe1ed9ae96 2023-02-11 03:53:55,704 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e2015112854b80465a8a6469b83b3c750c028fc65 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e2015112854b80465a8a6469b83b3c750c028fc65 2023-02-11 03:53:55,704 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthly 2 expired mob files are deleted 2023-02-11 03:53:55,704 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151101' 2023-02-11 03:53:55,705 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 2 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-02-11 03:53:55,705 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-02-11 03:53:55,705 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-02-11 03:53:55,705 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676087621935] compactions.PartitionedMobCompactor(354): After compaction, there are 4 mob files. table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-02-11 03:53:55,705 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676087621935] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-11 03:53:55,706 DEBUG [PEWorker-4] locking.LockProcedure(242): UNLOCKED pid=18, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE 2023-02-11 03:53:55,708 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=18, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE in 519 msec 2023-02-11 03:53:55,793 INFO [Listener at localhost/46205] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMajorCompactionWithWeeklyFollowedByMonthly Thread=246 (was 210) Potentially hanging thread: (jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:458) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.take(SynchronousQueue.java:924) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:53383@0x68fb9147-SendThread(127.0.0.1:53383) sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:345) org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1223) Potentially hanging thread: hfile_cleaner-dir-scan-pool-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: htable-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS:0;jenkins-hbase4:37011-shortCompactions-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.PriorityBlockingQueue.take(PriorityBlockingQueue.java:549) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-4-3 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:883) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_117507140_15 at /127.0.0.1:45014 [Waiting for operation #12] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_117507140_15 at /127.0.0.1:44958 [Waiting for operation #12] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1462371060_15 at /127.0.0.1:44998 [Waiting for operation #8] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_117507140_15 at /127.0.0.1:45004 [Waiting for operation #9] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_117507140_15 at /127.0.0.1:44858 [Waiting for operation #13] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x516e823e-shared-pool-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:53383@0x7228ad49 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.DelayQueue.poll(DelayQueue.java:259) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient.run(ReadOnlyZKClient.java:324) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient$$Lambda$45/1251065594.run(Unknown Source) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x516e823e-shared-pool-3 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:53383@0x7228ad49-SendThread(127.0.0.1:53383) sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:345) org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1223) Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/3712ba8e-7346-8b4d-3d93-a55f0db4c73d/cluster_d15eb7ef-e1c7-b786-c58e-de5925629f9c/dfs/data/data2/current sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_117507140_15 at /127.0.0.1:44928 [Waiting for operation #10] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x4d5a23e3-shared-pool-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: MobCompactor-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-1-3 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676087621935 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-4-2 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:883) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hfile_cleaner-dir-scan-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:53383@0x68fb9147 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.DelayQueue.poll(DelayQueue.java:259) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient.run(ReadOnlyZKClient.java:324) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient$$Lambda$45/1251065594.run(Unknown Source) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_117507140_15 at /127.0.0.1:44914 [Waiting for operation #12] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Timer for 'HBase' metrics system java.lang.Object.wait(Native Method) java.util.TimerThread.mainLoop(Timer.java:552) java.util.TimerThread.run(Timer.java:505) Potentially hanging thread: hconnection-0x4d5a23e3-shared-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_117507140_15 at /127.0.0.1:33576 [Waiting for operation #6] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_117507140_15 at /127.0.0.1:44970 [Waiting for operation #14] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_117507140_15 at /127.0.0.1:44962 [Waiting for operation #7] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_117507140_15 at /127.0.0.1:45016 [Waiting for operation #12] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_117507140_15 at /127.0.0.1:44886 [Waiting for operation #9] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RSProcedureDispatcher-pool-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x4d5a23e3-metaLookup-shared--pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:458) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.take(SynchronousQueue.java:924) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_117507140_15 at /127.0.0.1:45010 [Waiting for operation #14] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:53383@0x7228ad49-EventThread sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:53383@0x68fb9147-EventThread sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506) Potentially hanging thread: RPCClient-NioEventLoopGroup-4-4 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:883) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_117507140_15 at /127.0.0.1:44934 [Waiting for operation #22] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_117507140_15 at /127.0.0.1:33570 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=591 (was 475) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=227 (was 252), ProcessCount=168 (was 168), AvailableMemoryMB=3974 (was 4084) 2023-02-11 03:53:55,806 INFO [Listener at localhost/46205] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMajorCompactionWithMonthlyPolicy Thread=246, OpenFileDescriptor=591, MaxFileDescriptor=60000, SystemLoadAverage=227, ProcessCount=168, AvailableMemoryMB=3974 2023-02-11 03:53:55,809 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] master.HMaster$4(2102): Client=jenkins//172.31.14.131 create 'testMajorCompactionWithMonthlyPolicy', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'} 2023-02-11 03:53:55,810 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] procedure2.ProcedureExecutor(1029): Stored pid=19, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testMajorCompactionWithMonthlyPolicy 2023-02-11 03:53:55,812 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=19, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_PRE_OPERATION 2023-02-11 03:53:55,812 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] master.MasterRpcServices(697): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testMajorCompactionWithMonthlyPolicy" procId is: 19 2023-02-11 03:53:55,813 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=19, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testMajorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-11 03:53:55,813 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] master.MasterRpcServices(1227): Checking to see if procedure is done pid=19 2023-02-11 03:53:55,814 DEBUG [HFileArchiver-3] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/.tmp/data/default/testMajorCompactionWithMonthlyPolicy/7868535fe478faa7f98625b86eeccc35 2023-02-11 03:53:55,815 DEBUG [HFileArchiver-3] backup.HFileArchiver(153): Directory hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/.tmp/data/default/testMajorCompactionWithMonthlyPolicy/7868535fe478faa7f98625b86eeccc35 empty. 2023-02-11 03:53:55,815 DEBUG [HFileArchiver-3] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/.tmp/data/default/testMajorCompactionWithMonthlyPolicy/7868535fe478faa7f98625b86eeccc35 2023-02-11 03:53:55,816 DEBUG [PEWorker-5] procedure.DeleteTableProcedure(328): Archived testMajorCompactionWithMonthlyPolicy regions 2023-02-11 03:53:55,914 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] master.MasterRpcServices(1227): Checking to see if procedure is done pid=19 2023-02-11 03:53:56,115 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] master.MasterRpcServices(1227): Checking to see if procedure is done pid=19 2023-02-11 03:53:56,225 DEBUG [PEWorker-5] util.FSTableDescriptors(570): Wrote into hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/.tmp/data/default/testMajorCompactionWithMonthlyPolicy/.tabledesc/.tableinfo.0000000001 2023-02-11 03:53:56,227 INFO [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(7671): creating {ENCODED => 7868535fe478faa7f98625b86eeccc35, NAME => 'testMajorCompactionWithMonthlyPolicy,,1676087635809.7868535fe478faa7f98625b86eeccc35.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testMajorCompactionWithMonthlyPolicy', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'}, regionDir=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/.tmp 2023-02-11 03:53:56,417 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] master.MasterRpcServices(1227): Checking to see if procedure is done pid=19 2023-02-11 03:53:56,633 DEBUG [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(865): Instantiated testMajorCompactionWithMonthlyPolicy,,1676087635809.7868535fe478faa7f98625b86eeccc35.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-11 03:53:56,634 DEBUG [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1603): Closing 7868535fe478faa7f98625b86eeccc35, disabling compactions & flushes 2023-02-11 03:53:56,634 INFO [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1625): Closing region testMajorCompactionWithMonthlyPolicy,,1676087635809.7868535fe478faa7f98625b86eeccc35. 2023-02-11 03:53:56,634 DEBUG [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testMajorCompactionWithMonthlyPolicy,,1676087635809.7868535fe478faa7f98625b86eeccc35. 2023-02-11 03:53:56,634 DEBUG [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1713): Acquired close lock on testMajorCompactionWithMonthlyPolicy,,1676087635809.7868535fe478faa7f98625b86eeccc35. after waiting 0 ms 2023-02-11 03:53:56,634 DEBUG [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1723): Updates disabled for region testMajorCompactionWithMonthlyPolicy,,1676087635809.7868535fe478faa7f98625b86eeccc35. 2023-02-11 03:53:56,634 INFO [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1837): Closed testMajorCompactionWithMonthlyPolicy,,1676087635809.7868535fe478faa7f98625b86eeccc35. 2023-02-11 03:53:56,634 DEBUG [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1557): Region close journal for 7868535fe478faa7f98625b86eeccc35: Waiting for close lock at 1676087636634Disabling compacts and flushes for region at 1676087636634Disabling writes for close at 1676087636634Writing region close event to WAL at 1676087636634Closed at 1676087636634 2023-02-11 03:53:56,637 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=19, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testMajorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_ADD_TO_META 2023-02-11 03:53:56,638 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMajorCompactionWithMonthlyPolicy,,1676087635809.7868535fe478faa7f98625b86eeccc35.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1676087636638"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676087636638"}]},"ts":"1676087636638"} 2023-02-11 03:53:56,640 INFO [PEWorker-5] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-11 03:53:56,641 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=19, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testMajorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-11 03:53:56,641 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithMonthlyPolicy","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676087636641"}]},"ts":"1676087636641"} 2023-02-11 03:53:56,643 INFO [PEWorker-5] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithMonthlyPolicy, state=ENABLING in hbase:meta 2023-02-11 03:53:56,645 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=20, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithMonthlyPolicy, region=7868535fe478faa7f98625b86eeccc35, ASSIGN}] 2023-02-11 03:53:56,647 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=20, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithMonthlyPolicy, region=7868535fe478faa7f98625b86eeccc35, ASSIGN 2023-02-11 03:53:56,648 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=20, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithMonthlyPolicy, region=7868535fe478faa7f98625b86eeccc35, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,37011,1676087618428; forceNewPlan=false, retain=false 2023-02-11 03:53:56,799 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=20 updating hbase:meta row=7868535fe478faa7f98625b86eeccc35, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:53:56,799 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithMonthlyPolicy,,1676087635809.7868535fe478faa7f98625b86eeccc35.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1676087636799"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676087636799"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676087636799"}]},"ts":"1676087636799"} 2023-02-11 03:53:56,801 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=21, ppid=20, state=RUNNABLE; OpenRegionProcedure 7868535fe478faa7f98625b86eeccc35, server=jenkins-hbase4.apache.org,37011,1676087618428}] 2023-02-11 03:53:56,918 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] master.MasterRpcServices(1227): Checking to see if procedure is done pid=19 2023-02-11 03:53:56,957 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionWithMonthlyPolicy,,1676087635809.7868535fe478faa7f98625b86eeccc35. 2023-02-11 03:53:56,957 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 7868535fe478faa7f98625b86eeccc35, NAME => 'testMajorCompactionWithMonthlyPolicy,,1676087635809.7868535fe478faa7f98625b86eeccc35.', STARTKEY => '', ENDKEY => ''} 2023-02-11 03:53:56,958 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithMonthlyPolicy 7868535fe478faa7f98625b86eeccc35 2023-02-11 03:53:56,958 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testMajorCompactionWithMonthlyPolicy,,1676087635809.7868535fe478faa7f98625b86eeccc35.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-11 03:53:56,958 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 7868535fe478faa7f98625b86eeccc35 2023-02-11 03:53:56,958 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 7868535fe478faa7f98625b86eeccc35 2023-02-11 03:53:56,959 INFO [StoreOpener-7868535fe478faa7f98625b86eeccc35-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 7868535fe478faa7f98625b86eeccc35 2023-02-11 03:53:56,961 DEBUG [StoreOpener-7868535fe478faa7f98625b86eeccc35-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithMonthlyPolicy/7868535fe478faa7f98625b86eeccc35/family1 2023-02-11 03:53:56,961 DEBUG [StoreOpener-7868535fe478faa7f98625b86eeccc35-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithMonthlyPolicy/7868535fe478faa7f98625b86eeccc35/family1 2023-02-11 03:53:56,962 INFO [StoreOpener-7868535fe478faa7f98625b86eeccc35-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 7868535fe478faa7f98625b86eeccc35 columnFamilyName family1 2023-02-11 03:53:56,963 INFO [StoreOpener-7868535fe478faa7f98625b86eeccc35-1] regionserver.HStore(310): Store=7868535fe478faa7f98625b86eeccc35/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 03:53:56,964 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithMonthlyPolicy/7868535fe478faa7f98625b86eeccc35 2023-02-11 03:53:56,965 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithMonthlyPolicy/7868535fe478faa7f98625b86eeccc35 2023-02-11 03:53:56,967 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 7868535fe478faa7f98625b86eeccc35 2023-02-11 03:53:56,970 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithMonthlyPolicy/7868535fe478faa7f98625b86eeccc35/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-11 03:53:56,970 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 7868535fe478faa7f98625b86eeccc35; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=58733173, jitterRate=-0.12480752170085907}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-11 03:53:56,970 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 7868535fe478faa7f98625b86eeccc35: Running coprocessor pre-open hook at 1676087636958Writing region info on filesystem at 1676087636958Initializing all the Stores at 1676087636959 (+1 ms)Instantiating store for column family {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'} at 1676087636959Cleaning up temporary data from old regions at 1676087636965 (+6 ms)Cleaning up detritus from prior splits at 1676087636966 (+1 ms)Running coprocessor post-open hooks at 1676087636970 (+4 ms)Region opened successfully at 1676087636970 2023-02-11 03:53:56,971 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionWithMonthlyPolicy,,1676087635809.7868535fe478faa7f98625b86eeccc35., pid=21, masterSystemTime=1676087636953 2023-02-11 03:53:56,973 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionWithMonthlyPolicy,,1676087635809.7868535fe478faa7f98625b86eeccc35. 2023-02-11 03:53:56,973 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionWithMonthlyPolicy,,1676087635809.7868535fe478faa7f98625b86eeccc35. 2023-02-11 03:53:56,974 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=20 updating hbase:meta row=7868535fe478faa7f98625b86eeccc35, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:53:56,974 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithMonthlyPolicy,,1676087635809.7868535fe478faa7f98625b86eeccc35.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1676087636973"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676087636973"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676087636973"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676087636973"}]},"ts":"1676087636973"} 2023-02-11 03:53:56,977 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=21, resume processing ppid=20 2023-02-11 03:53:56,977 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=21, ppid=20, state=SUCCESS; OpenRegionProcedure 7868535fe478faa7f98625b86eeccc35, server=jenkins-hbase4.apache.org,37011,1676087618428 in 174 msec 2023-02-11 03:53:56,979 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=20, resume processing ppid=19 2023-02-11 03:53:56,979 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=20, ppid=19, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithMonthlyPolicy, region=7868535fe478faa7f98625b86eeccc35, ASSIGN in 332 msec 2023-02-11 03:53:56,980 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=19, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testMajorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-11 03:53:56,980 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithMonthlyPolicy","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676087636980"}]},"ts":"1676087636980"} 2023-02-11 03:53:56,982 INFO [PEWorker-2] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithMonthlyPolicy, state=ENABLED in hbase:meta 2023-02-11 03:53:56,984 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=19, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_POST_OPERATION 2023-02-11 03:53:56,986 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=19, state=SUCCESS; CreateTableProcedure table=testMajorCompactionWithMonthlyPolicy in 1.1750 sec 2023-02-11 03:53:57,919 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] master.MasterRpcServices(1227): Checking to see if procedure is done pid=19 2023-02-11 03:53:57,919 INFO [Listener at localhost/46205] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testMajorCompactionWithMonthlyPolicy, procId: 19 completed 2023-02-11 03:53:57,956 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 03:53:57,957 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testMajorCompactionWithMonthlyPolicy' 2023-02-11 03:53:57,958 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-02-11 03:53:57,958 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-02-11 03:53:57,958 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:53:57,959 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-02-11 03:53:57,959 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-02-11 03:53:57,959 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-11 03:53:57,959 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-11 03:53:57,962 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:53:57,962 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:53:57,962 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:53:57,962 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:53:57,962 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:53:57,962 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 03:53:57,963 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-11 03:53:57,963 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-11 03:53:57,963 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 03:53:57,963 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-11 03:53:57,963 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-02-11 03:53:57,964 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-02-11 03:53:57,964 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-02-11 03:53:57,964 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:53:57,964 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-02-11 03:53:57,965 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 03:53:57,965 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 03:53:57,965 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1676087635809.7868535fe478faa7f98625b86eeccc35. 2023-02-11 03:53:57,965 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1676087635809.7868535fe478faa7f98625b86eeccc35. started... 2023-02-11 03:53:57,965 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 7868535fe478faa7f98625b86eeccc35 1/1 column families, dataSize=74.11 KB heapSize=148.69 KB 2023-02-11 03:53:58,389 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201511309a80a4601ae1499e81f46a5bdcbcef85 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e201511309a80a4601ae1499e81f46a5bdcbcef85 2023-02-11 03:53:58,794 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=74.1 K, hasBloomFilter=true, into tmp file hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithMonthlyPolicy/7868535fe478faa7f98625b86eeccc35/.tmp/family1/90c940c0030a49f185d32ff2bc961ee5 2023-02-11 03:53:58,800 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithMonthlyPolicy/7868535fe478faa7f98625b86eeccc35/.tmp/family1/90c940c0030a49f185d32ff2bc961ee5 as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithMonthlyPolicy/7868535fe478faa7f98625b86eeccc35/family1/90c940c0030a49f185d32ff2bc961ee5 2023-02-11 03:53:58,806 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithMonthlyPolicy/7868535fe478faa7f98625b86eeccc35/family1/90c940c0030a49f185d32ff2bc961ee5, entries=1000, sequenceid=5, filesize=166.0 K 2023-02-11 03:53:58,807 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~74.11 KB/75890, heapSize ~148.67 KB/152240, currentSize=0 B/0 for 7868535fe478faa7f98625b86eeccc35 in 842ms, sequenceid=5, compaction requested=false 2023-02-11 03:53:58,808 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 7868535fe478faa7f98625b86eeccc35: 2023-02-11 03:53:58,808 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1676087635809.7868535fe478faa7f98625b86eeccc35. 2023-02-11 03:53:58,808 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 03:53:58,808 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 03:53:58,808 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:53:58,808 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-02-11 03:53:58,808 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37011,1676087618428' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-02-11 03:53:58,810 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-11 03:53:58,810 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:53:58,810 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:53:58,810 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:53:58,810 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:53:58,810 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-11 03:53:58,810 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 03:53:58,810 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:53:58,810 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:53:58,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-11 03:53:58,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:53:58,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:53:58,811 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37011,1676087618428' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-02-11 03:53:58,811 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7100c3b0[Count = 0] remaining members to acquire global barrier 2023-02-11 03:53:58,811 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-02-11 03:53:58,811 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-11 03:53:58,813 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-11 03:53:58,813 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-11 03:53:58,813 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-11 03:53:58,814 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-02-11 03:53:58,814 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-02-11 03:53:58,814 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:53:58,814 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 03:53:58,814 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,37011,1676087618428' in zk 2023-02-11 03:53:58,815 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:53:58,815 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-02-11 03:53:58,815 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:53:58,815 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:53:58,815 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:53:58,815 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:53:58,815 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-02-11 03:53:58,816 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:53:58,816 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:53:58,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-11 03:53:58,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:53:58,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:53:58,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-11 03:53:58,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:53:58,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,37011,1676087618428': 2023-02-11 03:53:58,818 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37011,1676087618428' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-02-11 03:53:58,818 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-02-11 03:53:58,818 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 03:53:58,818 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 03:53:58,818 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-02-11 03:53:58,818 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 03:53:58,819 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-11 03:53:58,819 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-11 03:53:58,820 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-11 03:53:58,820 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:53:58,820 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:53:58,820 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-11 03:53:58,820 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-11 03:53:58,820 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:53:58,820 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:53:58,820 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:53:58,820 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:53:58,820 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:53:58,820 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-11 03:53:58,820 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-11 03:53:58,821 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:53:58,821 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-11 03:53:58,821 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:53:58,821 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:53:58,822 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:53:58,822 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-11 03:53:58,822 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:53:58,823 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:53:58,823 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:53:58,824 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-11 03:53:58,824 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:53:58,824 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-02-11 03:53:58,824 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:53:58,824 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-02-11 03:53:58,824 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 03:53:58,824 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-11 03:53:58,824 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:53:58,824 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:53:58,824 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:53:58,825 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:53:58,825 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:53:58,825 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-11 03:53:58,825 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-02-11 03:53:58,825 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 03:53:58,825 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-11 03:53:58,825 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-11 03:53:58,925 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 03:53:58,926 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 03:53:58,932 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 03:53:58,936 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-02-11 03:53:58,936 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-02-11 03:53:58,936 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-02-11 03:53:58,936 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:53:58,937 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-02-11 03:53:58,937 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-02-11 03:53:58,937 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-11 03:53:58,938 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-11 03:53:58,940 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:53:58,940 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:53:58,940 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:53:58,941 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:53:58,941 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:53:58,941 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 03:53:58,941 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-11 03:53:58,941 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-11 03:53:58,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 03:53:58,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-11 03:53:58,942 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-02-11 03:53:58,942 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-02-11 03:53:58,942 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-02-11 03:53:58,942 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-02-11 03:53:58,942 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:53:58,943 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-02-11 03:53:58,943 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 03:53:58,943 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 03:53:58,943 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1676087635809.7868535fe478faa7f98625b86eeccc35. 2023-02-11 03:53:58,943 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1676087635809.7868535fe478faa7f98625b86eeccc35. started... 2023-02-11 03:53:58,943 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 7868535fe478faa7f98625b86eeccc35 1/1 column families, dataSize=74 B heapSize=408 B 2023-02-11 03:53:58,975 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151128ea3aaa134ade4bf9b7e67c19e9ee20ef to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151128ea3aaa134ade4bf9b7e67c19e9ee20ef 2023-02-11 03:53:58,989 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=74, hasBloomFilter=true, into tmp file hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithMonthlyPolicy/7868535fe478faa7f98625b86eeccc35/.tmp/family1/a384360e394e46bb8e08957a76677412 2023-02-11 03:53:58,997 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithMonthlyPolicy/7868535fe478faa7f98625b86eeccc35/.tmp/family1/a384360e394e46bb8e08957a76677412 as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithMonthlyPolicy/7868535fe478faa7f98625b86eeccc35/family1/a384360e394e46bb8e08957a76677412 2023-02-11 03:53:59,003 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithMonthlyPolicy/7868535fe478faa7f98625b86eeccc35/family1/a384360e394e46bb8e08957a76677412, entries=1, sequenceid=9, filesize=5.0 K 2023-02-11 03:53:59,004 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~74 B/74, heapSize ~392 B/392, currentSize=0 B/0 for 7868535fe478faa7f98625b86eeccc35 in 61ms, sequenceid=9, compaction requested=false 2023-02-11 03:53:59,004 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 7868535fe478faa7f98625b86eeccc35: 2023-02-11 03:53:59,004 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1676087635809.7868535fe478faa7f98625b86eeccc35. 2023-02-11 03:53:59,004 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 03:53:59,004 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 03:53:59,004 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:53:59,004 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-02-11 03:53:59,004 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37011,1676087618428' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-02-11 03:53:59,006 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:53:59,006 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-11 03:53:59,006 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:53:59,006 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:53:59,006 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:53:59,006 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-11 03:53:59,006 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 03:53:59,007 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:53:59,007 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:53:59,007 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-11 03:53:59,007 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:53:59,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:53:59,008 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37011,1676087618428' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-02-11 03:53:59,008 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7512582b[Count = 0] remaining members to acquire global barrier 2023-02-11 03:53:59,008 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-02-11 03:53:59,008 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-11 03:53:59,009 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-11 03:53:59,010 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-11 03:53:59,010 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-11 03:53:59,010 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-02-11 03:53:59,010 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-02-11 03:53:59,010 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,37011,1676087618428' in zk 2023-02-11 03:53:59,010 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:53:59,010 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 03:53:59,011 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:53:59,011 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-02-11 03:53:59,011 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:53:59,011 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:53:59,011 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:53:59,011 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:53:59,011 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-02-11 03:53:59,012 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:53:59,013 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:53:59,013 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-11 03:53:59,013 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:53:59,014 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:53:59,014 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-11 03:53:59,014 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:53:59,015 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,37011,1676087618428': 2023-02-11 03:53:59,015 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37011,1676087618428' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-02-11 03:53:59,015 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-02-11 03:53:59,015 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 03:53:59,015 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 03:53:59,015 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-02-11 03:53:59,015 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 03:53:59,017 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-11 03:53:59,017 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-11 03:53:59,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-11 03:53:59,018 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-11 03:53:59,018 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:53:59,018 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:53:59,018 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:53:59,018 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-11 03:53:59,018 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:53:59,018 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:53:59,018 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:53:59,018 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:53:59,018 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-11 03:53:59,019 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-11 03:53:59,019 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:53:59,019 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-11 03:53:59,019 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:53:59,020 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:53:59,020 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:53:59,020 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-11 03:53:59,020 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:53:59,027 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:53:59,027 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:53:59,027 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-11 03:53:59,027 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:53:59,027 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-11 03:53:59,027 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-02-11 03:53:59,028 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 03:53:59,027 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:53:59,028 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:53:59,028 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:53:59,027 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:53:59,028 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-11 03:53:59,028 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-11 03:53:59,028 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-11 03:53:59,028 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-02-11 03:53:59,029 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 03:53:59,029 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:53:59,029 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:53:59,129 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 03:53:59,130 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 03:53:59,135 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 03:53:59,137 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-02-11 03:53:59,137 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-02-11 03:53:59,137 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-02-11 03:53:59,137 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:53:59,138 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-02-11 03:53:59,138 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-02-11 03:53:59,138 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-11 03:53:59,138 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-11 03:53:59,140 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:53:59,140 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:53:59,140 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:53:59,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:53:59,140 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:53:59,140 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 03:53:59,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-11 03:53:59,141 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-11 03:53:59,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 03:53:59,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-11 03:53:59,141 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-02-11 03:53:59,141 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-02-11 03:53:59,141 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-02-11 03:53:59,141 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-02-11 03:53:59,141 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:53:59,142 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-02-11 03:53:59,142 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 03:53:59,142 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 03:53:59,142 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1676087635809.7868535fe478faa7f98625b86eeccc35. 2023-02-11 03:53:59,142 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1676087635809.7868535fe478faa7f98625b86eeccc35. started... 2023-02-11 03:53:59,142 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 7868535fe478faa7f98625b86eeccc35 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-11 03:53:59,559 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151201540ad5143d064dcea31b1b3ae061e981 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151201540ad5143d064dcea31b1b3ae061e981 2023-02-11 03:53:59,967 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithMonthlyPolicy/7868535fe478faa7f98625b86eeccc35/.tmp/family1/908cdfb4203c4d70989c76de1fe2bda2 2023-02-11 03:53:59,975 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithMonthlyPolicy/7868535fe478faa7f98625b86eeccc35/.tmp/family1/908cdfb4203c4d70989c76de1fe2bda2 as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithMonthlyPolicy/7868535fe478faa7f98625b86eeccc35/family1/908cdfb4203c4d70989c76de1fe2bda2 2023-02-11 03:53:59,982 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithMonthlyPolicy/7868535fe478faa7f98625b86eeccc35/family1/908cdfb4203c4d70989c76de1fe2bda2, entries=1, sequenceid=13, filesize=4.9 K 2023-02-11 03:53:59,983 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 7868535fe478faa7f98625b86eeccc35 in 841ms, sequenceid=13, compaction requested=true 2023-02-11 03:53:59,983 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 7868535fe478faa7f98625b86eeccc35: 2023-02-11 03:53:59,983 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1676087635809.7868535fe478faa7f98625b86eeccc35. 2023-02-11 03:53:59,983 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 03:53:59,983 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 03:53:59,983 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:53:59,983 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-02-11 03:53:59,983 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37011,1676087618428' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-02-11 03:53:59,986 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-11 03:53:59,986 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:53:59,986 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:53:59,986 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:53:59,986 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:53:59,986 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-11 03:53:59,986 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 03:53:59,986 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:53:59,987 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:53:59,987 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-11 03:53:59,987 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:53:59,988 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:53:59,988 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37011,1676087618428' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-02-11 03:53:59,988 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@247e8aa6[Count = 0] remaining members to acquire global barrier 2023-02-11 03:53:59,988 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-02-11 03:53:59,988 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-11 03:53:59,990 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-11 03:53:59,990 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-11 03:53:59,990 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-11 03:53:59,990 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-02-11 03:53:59,990 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-02-11 03:53:59,990 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,37011,1676087618428' in zk 2023-02-11 03:53:59,991 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:53:59,991 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 03:53:59,993 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-02-11 03:53:59,993 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:53:59,993 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:53:59,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:53:59,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:53:59,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:53:59,993 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-02-11 03:53:59,995 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:53:59,995 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:53:59,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-11 03:53:59,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:53:59,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:53:59,997 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-11 03:53:59,997 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:53:59,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,37011,1676087618428': 2023-02-11 03:53:59,998 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37011,1676087618428' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-02-11 03:53:59,998 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-02-11 03:53:59,998 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 03:53:59,998 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 03:53:59,998 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-02-11 03:53:59,998 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 03:54:00,000 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-11 03:54:00,000 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-11 03:54:00,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-11 03:54:00,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:54:00,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:54:00,000 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:00,000 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-11 03:54:00,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-11 03:54:00,001 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:00,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:00,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:54:00,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-11 03:54:00,002 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:00,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-11 03:54:00,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:54:00,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-11 03:54:00,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:00,003 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:00,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:54:00,004 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-11 03:54:00,004 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:00,007 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:00,007 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:00,007 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:00,007 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:00,007 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:00,007 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-11 03:54:00,007 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-11 03:54:00,007 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:54:00,007 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:00,008 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-11 03:54:00,008 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-11 03:54:00,008 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-11 03:54:00,007 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-02-11 03:54:00,009 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 03:54:00,009 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:00,009 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:00,009 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-02-11 03:54:00,009 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 03:54:00,109 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 03:54:00,111 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 03:54:00,117 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 03:54:00,119 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-02-11 03:54:00,119 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-02-11 03:54:00,119 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-02-11 03:54:00,119 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:54:00,120 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-02-11 03:54:00,120 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-02-11 03:54:00,121 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-11 03:54:00,121 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-11 03:54:00,122 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:00,122 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:00,122 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:00,122 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:00,122 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:00,122 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 03:54:00,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-11 03:54:00,123 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-11 03:54:00,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 03:54:00,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-11 03:54:00,123 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-02-11 03:54:00,124 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-02-11 03:54:00,124 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-02-11 03:54:00,124 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-02-11 03:54:00,124 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:54:00,124 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-02-11 03:54:00,124 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 03:54:00,124 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 03:54:00,124 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1676087635809.7868535fe478faa7f98625b86eeccc35. 2023-02-11 03:54:00,125 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1676087635809.7868535fe478faa7f98625b86eeccc35. started... 2023-02-11 03:54:00,125 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 7868535fe478faa7f98625b86eeccc35 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-11 03:54:00,149 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151205f21209223499465fa3cc251b0c322a59 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151205f21209223499465fa3cc251b0c322a59 2023-02-11 03:54:00,163 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithMonthlyPolicy/7868535fe478faa7f98625b86eeccc35/.tmp/family1/935d7e82ff354caa84abfd45b607faeb 2023-02-11 03:54:00,173 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithMonthlyPolicy/7868535fe478faa7f98625b86eeccc35/.tmp/family1/935d7e82ff354caa84abfd45b607faeb as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithMonthlyPolicy/7868535fe478faa7f98625b86eeccc35/family1/935d7e82ff354caa84abfd45b607faeb 2023-02-11 03:54:00,180 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithMonthlyPolicy/7868535fe478faa7f98625b86eeccc35/family1/935d7e82ff354caa84abfd45b607faeb, entries=1, sequenceid=17, filesize=4.9 K 2023-02-11 03:54:00,182 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 7868535fe478faa7f98625b86eeccc35 in 57ms, sequenceid=17, compaction requested=true 2023-02-11 03:54:00,182 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 7868535fe478faa7f98625b86eeccc35: 2023-02-11 03:54:00,182 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1676087635809.7868535fe478faa7f98625b86eeccc35. 2023-02-11 03:54:00,182 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 03:54:00,182 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 03:54:00,182 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:00,182 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-02-11 03:54:00,182 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37011,1676087618428' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-02-11 03:54:00,184 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:00,184 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-11 03:54:00,185 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:00,185 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:54:00,185 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:54:00,185 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-11 03:54:00,185 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 03:54:00,185 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:54:00,185 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:54:00,186 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-11 03:54:00,186 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:00,186 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:54:00,187 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37011,1676087618428' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-02-11 03:54:00,187 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-02-11 03:54:00,187 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@722bbe16[Count = 0] remaining members to acquire global barrier 2023-02-11 03:54:00,187 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-11 03:54:00,188 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-11 03:54:00,188 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-11 03:54:00,188 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-11 03:54:00,188 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-02-11 03:54:00,188 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-02-11 03:54:00,188 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:00,189 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 03:54:00,188 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,37011,1676087618428' in zk 2023-02-11 03:54:00,191 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-02-11 03:54:00,191 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:00,191 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:54:00,191 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:00,191 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-02-11 03:54:00,191 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:54:00,192 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:54:00,192 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:54:00,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:54:00,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-11 03:54:00,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:00,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:54:00,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-11 03:54:00,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:00,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,37011,1676087618428': 2023-02-11 03:54:00,195 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37011,1676087618428' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-02-11 03:54:00,195 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-02-11 03:54:00,195 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 03:54:00,195 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 03:54:00,195 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-02-11 03:54:00,195 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 03:54:00,197 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-11 03:54:00,197 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-11 03:54:00,197 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-11 03:54:00,197 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:54:00,197 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:54:00,197 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-11 03:54:00,197 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:00,197 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-11 03:54:00,198 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:54:00,198 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:00,198 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:00,198 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:00,198 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-11 03:54:00,198 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-11 03:54:00,199 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:54:00,199 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-11 03:54:00,199 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:00,199 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:00,200 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:54:00,200 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-11 03:54:00,200 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:00,202 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:00,202 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:00,202 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:00,202 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:00,202 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-02-11 03:54:00,202 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 03:54:00,202 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:00,202 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:54:00,202 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-11 03:54:00,203 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-11 03:54:00,203 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:00,203 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-11 03:54:00,203 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-11 03:54:00,203 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:00,203 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-11 03:54:00,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:00,203 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-02-11 03:54:00,204 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 03:54:00,304 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 03:54:00,305 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 03:54:00,310 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 03:54:00,312 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-02-11 03:54:00,313 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-02-11 03:54:00,313 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-02-11 03:54:00,313 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:54:00,314 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-02-11 03:54:00,314 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-02-11 03:54:00,314 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-11 03:54:00,314 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-11 03:54:00,315 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:00,315 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:00,316 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:00,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:00,316 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:00,316 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 03:54:00,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-11 03:54:00,317 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-11 03:54:00,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 03:54:00,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-11 03:54:00,317 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-02-11 03:54:00,317 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-02-11 03:54:00,317 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-02-11 03:54:00,318 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-02-11 03:54:00,318 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:54:00,318 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-02-11 03:54:00,318 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 03:54:00,318 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 03:54:00,318 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1676087635809.7868535fe478faa7f98625b86eeccc35. 2023-02-11 03:54:00,318 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1676087635809.7868535fe478faa7f98625b86eeccc35. started... 2023-02-11 03:54:00,319 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 7868535fe478faa7f98625b86eeccc35 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-11 03:54:00,366 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-11 03:54:00,736 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512289a05e206db7a4f46a6f1f6512b0d3bfb to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e201512289a05e206db7a4f46a6f1f6512b0d3bfb 2023-02-11 03:54:01,143 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=21, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithMonthlyPolicy/7868535fe478faa7f98625b86eeccc35/.tmp/family1/885ef12989694ea09b894d857471f116 2023-02-11 03:54:01,150 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithMonthlyPolicy/7868535fe478faa7f98625b86eeccc35/.tmp/family1/885ef12989694ea09b894d857471f116 as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithMonthlyPolicy/7868535fe478faa7f98625b86eeccc35/family1/885ef12989694ea09b894d857471f116 2023-02-11 03:54:01,155 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithMonthlyPolicy/7868535fe478faa7f98625b86eeccc35/family1/885ef12989694ea09b894d857471f116, entries=1, sequenceid=21, filesize=4.9 K 2023-02-11 03:54:01,157 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 7868535fe478faa7f98625b86eeccc35 in 838ms, sequenceid=21, compaction requested=true 2023-02-11 03:54:01,157 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 7868535fe478faa7f98625b86eeccc35: 2023-02-11 03:54:01,157 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1676087635809.7868535fe478faa7f98625b86eeccc35. 2023-02-11 03:54:01,157 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 03:54:01,157 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 03:54:01,157 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:01,157 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-02-11 03:54:01,157 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37011,1676087618428' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-02-11 03:54:01,159 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-11 03:54:01,159 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:01,159 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:01,159 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:54:01,159 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:54:01,159 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-11 03:54:01,159 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 03:54:01,159 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:54:01,160 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:54:01,160 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-11 03:54:01,160 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:01,160 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:54:01,161 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37011,1676087618428' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-02-11 03:54:01,161 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@541eea0[Count = 0] remaining members to acquire global barrier 2023-02-11 03:54:01,161 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-02-11 03:54:01,161 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-11 03:54:01,162 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-11 03:54:01,162 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-11 03:54:01,162 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-11 03:54:01,163 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-02-11 03:54:01,163 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:01,163 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 03:54:01,163 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-02-11 03:54:01,163 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,37011,1676087618428' in zk 2023-02-11 03:54:01,166 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-02-11 03:54:01,166 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:01,166 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:54:01,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:01,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:54:01,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:54:01,166 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-02-11 03:54:01,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:54:01,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:54:01,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-11 03:54:01,168 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:01,168 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:54:01,168 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-11 03:54:01,168 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:01,169 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,37011,1676087618428': 2023-02-11 03:54:01,169 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37011,1676087618428' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-02-11 03:54:01,169 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-02-11 03:54:01,169 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 03:54:01,169 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 03:54:01,169 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-02-11 03:54:01,169 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 03:54:01,174 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-11 03:54:01,174 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-11 03:54:01,174 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-11 03:54:01,174 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-11 03:54:01,174 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:54:01,174 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:54:01,174 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:01,174 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-11 03:54:01,175 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:01,175 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:01,175 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:54:01,175 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:01,175 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-11 03:54:01,175 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-11 03:54:01,176 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:54:01,176 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-11 03:54:01,176 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:01,176 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:01,176 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:54:01,177 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-11 03:54:01,177 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:01,178 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:01,179 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-11 03:54:01,179 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-11 03:54:01,179 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-02-11 03:54:01,179 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 03:54:01,179 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:01,179 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:01,179 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:01,179 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:01,179 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-02-11 03:54:01,179 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 03:54:01,179 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-02-11 03:54:01,179 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:54:01,179 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-11 03:54:01,179 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:01,179 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-11 03:54:01,180 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-11 03:54:01,180 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:01,180 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:01,279 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 03:54:01,280 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 03:54:01,286 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 03:54:01,288 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-02-11 03:54:01,288 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-02-11 03:54:01,288 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-02-11 03:54:01,288 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:54:01,289 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-02-11 03:54:01,289 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-02-11 03:54:01,290 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-11 03:54:01,290 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-11 03:54:01,291 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:01,291 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:01,291 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:01,291 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:01,292 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:01,292 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 03:54:01,292 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-11 03:54:01,292 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-11 03:54:01,293 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 03:54:01,293 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-11 03:54:01,293 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-02-11 03:54:01,293 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-02-11 03:54:01,293 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-02-11 03:54:01,293 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-02-11 03:54:01,293 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:54:01,294 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-02-11 03:54:01,294 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 03:54:01,294 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 03:54:01,294 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1676087635809.7868535fe478faa7f98625b86eeccc35. 2023-02-11 03:54:01,294 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1676087635809.7868535fe478faa7f98625b86eeccc35. started... 2023-02-11 03:54:01,294 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 7868535fe478faa7f98625b86eeccc35 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-11 03:54:01,712 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512319380924a0cb94a60a2d7cd843cbedaae to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e201512319380924a0cb94a60a2d7cd843cbedaae 2023-02-11 03:54:02,118 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=25, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithMonthlyPolicy/7868535fe478faa7f98625b86eeccc35/.tmp/family1/821e350aedc44d50b73ec1cf0eb1706a 2023-02-11 03:54:02,127 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithMonthlyPolicy/7868535fe478faa7f98625b86eeccc35/.tmp/family1/821e350aedc44d50b73ec1cf0eb1706a as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithMonthlyPolicy/7868535fe478faa7f98625b86eeccc35/family1/821e350aedc44d50b73ec1cf0eb1706a 2023-02-11 03:54:02,133 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithMonthlyPolicy/7868535fe478faa7f98625b86eeccc35/family1/821e350aedc44d50b73ec1cf0eb1706a, entries=1, sequenceid=25, filesize=4.9 K 2023-02-11 03:54:02,135 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 7868535fe478faa7f98625b86eeccc35 in 841ms, sequenceid=25, compaction requested=true 2023-02-11 03:54:02,135 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 7868535fe478faa7f98625b86eeccc35: 2023-02-11 03:54:02,135 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1676087635809.7868535fe478faa7f98625b86eeccc35. 2023-02-11 03:54:02,135 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 03:54:02,135 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 03:54:02,135 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:02,135 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-02-11 03:54:02,135 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37011,1676087618428' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-02-11 03:54:02,137 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:02,137 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-11 03:54:02,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:02,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:54:02,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:54:02,137 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-11 03:54:02,137 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 03:54:02,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:54:02,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:54:02,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-11 03:54:02,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:02,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:54:02,139 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37011,1676087618428' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-02-11 03:54:02,139 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@62753d23[Count = 0] remaining members to acquire global barrier 2023-02-11 03:54:02,139 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-02-11 03:54:02,140 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-11 03:54:02,141 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-11 03:54:02,141 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-11 03:54:02,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-11 03:54:02,141 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-02-11 03:54:02,141 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-02-11 03:54:02,141 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:02,141 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 03:54:02,141 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,37011,1676087618428' in zk 2023-02-11 03:54:02,144 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:02,144 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-02-11 03:54:02,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:02,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:54:02,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:54:02,144 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:54:02,144 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-02-11 03:54:02,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:54:02,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:54:02,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-11 03:54:02,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:02,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:54:02,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-11 03:54:02,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:02,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,37011,1676087618428': 2023-02-11 03:54:02,148 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37011,1676087618428' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-02-11 03:54:02,148 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-02-11 03:54:02,148 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 03:54:02,148 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 03:54:02,148 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-02-11 03:54:02,148 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 03:54:02,149 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-11 03:54:02,149 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-11 03:54:02,149 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-11 03:54:02,150 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-11 03:54:02,150 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:54:02,150 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:54:02,150 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-11 03:54:02,149 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:02,150 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:54:02,150 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:02,150 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:02,150 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:02,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-11 03:54:02,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-11 03:54:02,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:54:02,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-11 03:54:02,152 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:02,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:02,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:54:02,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-11 03:54:02,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:02,157 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:02,157 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:02,157 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:02,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:02,157 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:02,158 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:54:02,158 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-02-11 03:54:02,158 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 03:54:02,158 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-11 03:54:02,158 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-11 03:54:02,158 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:02,158 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-11 03:54:02,159 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-11 03:54:02,159 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-11 03:54:02,159 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-02-11 03:54:02,159 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:02,159 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:02,159 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 03:54:02,259 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 03:54:02,261 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 03:54:02,266 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 03:54:02,268 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-02-11 03:54:02,268 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-02-11 03:54:02,268 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-02-11 03:54:02,268 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:54:02,269 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-02-11 03:54:02,269 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-02-11 03:54:02,269 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-11 03:54:02,269 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-11 03:54:02,270 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:02,271 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:02,271 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:02,271 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:02,271 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:02,271 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 03:54:02,271 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-11 03:54:02,271 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-11 03:54:02,272 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 03:54:02,272 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-11 03:54:02,272 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-02-11 03:54:02,272 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-02-11 03:54:02,272 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-02-11 03:54:02,272 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-02-11 03:54:02,272 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:54:02,273 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-02-11 03:54:02,276 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 03:54:02,276 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 03:54:02,276 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1676087635809.7868535fe478faa7f98625b86eeccc35. 2023-02-11 03:54:02,276 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1676087635809.7868535fe478faa7f98625b86eeccc35. started... 2023-02-11 03:54:02,276 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 7868535fe478faa7f98625b86eeccc35 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-11 03:54:02,709 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160101988b93ce6a8f4b70b854458640334685 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20160101988b93ce6a8f4b70b854458640334685 2023-02-11 03:54:02,716 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=29, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithMonthlyPolicy/7868535fe478faa7f98625b86eeccc35/.tmp/family1/bfe6fd6db6434ca3b905802733575006 2023-02-11 03:54:02,722 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithMonthlyPolicy/7868535fe478faa7f98625b86eeccc35/.tmp/family1/bfe6fd6db6434ca3b905802733575006 as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithMonthlyPolicy/7868535fe478faa7f98625b86eeccc35/family1/bfe6fd6db6434ca3b905802733575006 2023-02-11 03:54:02,728 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithMonthlyPolicy/7868535fe478faa7f98625b86eeccc35/family1/bfe6fd6db6434ca3b905802733575006, entries=1, sequenceid=29, filesize=4.9 K 2023-02-11 03:54:02,729 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 7868535fe478faa7f98625b86eeccc35 in 453ms, sequenceid=29, compaction requested=true 2023-02-11 03:54:02,729 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 7868535fe478faa7f98625b86eeccc35: 2023-02-11 03:54:02,729 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1676087635809.7868535fe478faa7f98625b86eeccc35. 2023-02-11 03:54:02,729 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 03:54:02,729 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 03:54:02,730 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:02,730 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-02-11 03:54:02,730 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37011,1676087618428' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-02-11 03:54:02,731 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:02,731 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:02,731 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-11 03:54:02,732 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:54:02,732 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:54:02,732 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:54:02,732 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-11 03:54:02,732 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 03:54:02,732 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:54:02,733 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-11 03:54:02,733 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:02,733 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:54:02,733 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37011,1676087618428' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-02-11 03:54:02,733 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@328eb071[Count = 0] remaining members to acquire global barrier 2023-02-11 03:54:02,733 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-02-11 03:54:02,734 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-11 03:54:02,735 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-11 03:54:02,735 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-11 03:54:02,735 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-11 03:54:02,735 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-02-11 03:54:02,735 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:02,735 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 03:54:02,735 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-02-11 03:54:02,735 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,37011,1676087618428' in zk 2023-02-11 03:54:02,737 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:02,737 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-02-11 03:54:02,737 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:02,737 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:54:02,737 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:54:02,737 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:54:02,738 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-02-11 03:54:02,738 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:54:02,739 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:54:02,739 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-11 03:54:02,739 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:02,739 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:54:02,740 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-11 03:54:02,740 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:02,740 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,37011,1676087618428': 2023-02-11 03:54:02,740 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37011,1676087618428' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-02-11 03:54:02,740 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-02-11 03:54:02,740 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 03:54:02,740 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 03:54:02,740 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-02-11 03:54:02,741 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 03:54:02,742 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-11 03:54:02,742 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-11 03:54:02,742 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-11 03:54:02,742 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-11 03:54:02,742 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:54:02,743 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:54:02,742 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:02,742 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-11 03:54:02,743 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:02,743 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:54:02,743 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:02,743 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:02,743 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-11 03:54:02,743 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-11 03:54:02,744 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:54:02,744 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-11 03:54:02,744 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:02,745 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:02,745 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:54:02,745 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-11 03:54:02,746 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:02,747 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:02,747 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:02,747 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:54:02,747 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:02,747 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:02,747 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-02-11 03:54:02,748 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 03:54:02,747 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-11 03:54:02,747 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:02,748 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-11 03:54:02,748 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:02,748 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-11 03:54:02,748 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-11 03:54:02,748 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-11 03:54:02,748 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-02-11 03:54:02,748 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 03:54:02,753 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-02-11 03:54:02,753 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:02,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:02,848 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 03:54:02,850 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 03:54:02,856 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 03:54:02,859 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-02-11 03:54:02,859 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-02-11 03:54:02,859 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-02-11 03:54:02,859 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:54:02,860 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-02-11 03:54:02,860 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-02-11 03:54:02,861 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-11 03:54:02,861 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-11 03:54:02,862 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:02,862 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:02,862 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:02,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:02,862 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:02,862 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 03:54:02,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-11 03:54:02,863 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-11 03:54:02,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 03:54:02,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-11 03:54:02,863 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-02-11 03:54:02,864 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-02-11 03:54:02,864 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-02-11 03:54:02,864 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-02-11 03:54:02,864 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:54:02,864 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-02-11 03:54:02,864 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 03:54:02,864 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 03:54:02,864 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1676087635809.7868535fe478faa7f98625b86eeccc35. 2023-02-11 03:54:02,865 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1676087635809.7868535fe478faa7f98625b86eeccc35. started... 2023-02-11 03:54:02,865 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 7868535fe478faa7f98625b86eeccc35 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-11 03:54:02,894 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201601036c675b7845d54ab39de821cb307469ea to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e201601036c675b7845d54ab39de821cb307469ea 2023-02-11 03:54:03,305 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=33, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithMonthlyPolicy/7868535fe478faa7f98625b86eeccc35/.tmp/family1/9e34002bac4847dc9e3ee80e2e09e33f 2023-02-11 03:54:03,311 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithMonthlyPolicy/7868535fe478faa7f98625b86eeccc35/.tmp/family1/9e34002bac4847dc9e3ee80e2e09e33f as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithMonthlyPolicy/7868535fe478faa7f98625b86eeccc35/family1/9e34002bac4847dc9e3ee80e2e09e33f 2023-02-11 03:54:03,316 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithMonthlyPolicy/7868535fe478faa7f98625b86eeccc35/family1/9e34002bac4847dc9e3ee80e2e09e33f, entries=1, sequenceid=33, filesize=4.9 K 2023-02-11 03:54:03,317 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 7868535fe478faa7f98625b86eeccc35 in 452ms, sequenceid=33, compaction requested=true 2023-02-11 03:54:03,317 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 7868535fe478faa7f98625b86eeccc35: 2023-02-11 03:54:03,317 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1676087635809.7868535fe478faa7f98625b86eeccc35. 2023-02-11 03:54:03,318 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 03:54:03,318 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 03:54:03,318 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:03,318 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-02-11 03:54:03,318 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37011,1676087618428' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-02-11 03:54:03,319 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-11 03:54:03,319 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:03,319 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:03,319 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:54:03,320 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:54:03,320 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-11 03:54:03,320 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 03:54:03,320 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:54:03,320 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:54:03,320 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-11 03:54:03,320 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:03,321 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:54:03,321 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37011,1676087618428' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-02-11 03:54:03,321 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1d7a6338[Count = 0] remaining members to acquire global barrier 2023-02-11 03:54:03,321 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-02-11 03:54:03,321 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-11 03:54:03,322 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-11 03:54:03,322 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-11 03:54:03,322 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-11 03:54:03,322 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-02-11 03:54:03,322 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-02-11 03:54:03,322 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,37011,1676087618428' in zk 2023-02-11 03:54:03,322 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:03,323 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 03:54:03,328 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-02-11 03:54:03,328 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:03,328 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:54:03,328 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:03,328 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:54:03,329 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:54:03,328 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-02-11 03:54:03,329 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:54:03,329 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:54:03,330 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-11 03:54:03,330 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:03,330 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:54:03,330 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-11 03:54:03,331 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:03,331 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,37011,1676087618428': 2023-02-11 03:54:03,331 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37011,1676087618428' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-02-11 03:54:03,331 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-02-11 03:54:03,331 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 03:54:03,331 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 03:54:03,331 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-02-11 03:54:03,331 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 03:54:03,333 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-11 03:54:03,333 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-11 03:54:03,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-11 03:54:03,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:54:03,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:54:03,333 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-11 03:54:03,333 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:03,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-11 03:54:03,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:54:03,333 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:03,333 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:03,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-11 03:54:03,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:03,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:54:03,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-11 03:54:03,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-11 03:54:03,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:03,335 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:03,335 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:54:03,335 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-11 03:54:03,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:03,337 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:03,337 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:03,337 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-11 03:54:03,337 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:54:03,337 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-02-11 03:54:03,337 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 03:54:03,337 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-11 03:54:03,337 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:03,338 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:03,337 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:03,338 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-02-11 03:54:03,338 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:03,338 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-02-11 03:54:03,338 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:03,338 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:03,338 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-11 03:54:03,338 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 03:54:03,338 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-11 03:54:03,338 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-11 03:54:03,438 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 03:54:03,440 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 03:54:03,444 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 03:54:03,446 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-02-11 03:54:03,447 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-02-11 03:54:03,447 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-02-11 03:54:03,447 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:54:03,448 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-02-11 03:54:03,448 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-02-11 03:54:03,448 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-11 03:54:03,448 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-11 03:54:03,450 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:03,450 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:03,450 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:03,450 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:03,450 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:03,450 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 03:54:03,450 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-11 03:54:03,450 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-11 03:54:03,451 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 03:54:03,451 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-11 03:54:03,451 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-02-11 03:54:03,451 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-02-11 03:54:03,451 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-02-11 03:54:03,451 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-02-11 03:54:03,451 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:54:03,452 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-02-11 03:54:03,452 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 03:54:03,452 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 03:54:03,452 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1676087635809.7868535fe478faa7f98625b86eeccc35. 2023-02-11 03:54:03,452 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1676087635809.7868535fe478faa7f98625b86eeccc35. started... 2023-02-11 03:54:03,452 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 7868535fe478faa7f98625b86eeccc35 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-11 03:54:03,869 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20150907da288471e0ee4a278b576abe1d957f2c to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20150907da288471e0ee4a278b576abe1d957f2c 2023-02-11 03:54:03,876 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=37, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithMonthlyPolicy/7868535fe478faa7f98625b86eeccc35/.tmp/family1/9b85eda0ae454bfb821810b73c8a6124 2023-02-11 03:54:03,882 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithMonthlyPolicy/7868535fe478faa7f98625b86eeccc35/.tmp/family1/9b85eda0ae454bfb821810b73c8a6124 as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithMonthlyPolicy/7868535fe478faa7f98625b86eeccc35/family1/9b85eda0ae454bfb821810b73c8a6124 2023-02-11 03:54:03,887 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithMonthlyPolicy/7868535fe478faa7f98625b86eeccc35/family1/9b85eda0ae454bfb821810b73c8a6124, entries=1, sequenceid=37, filesize=4.9 K 2023-02-11 03:54:03,888 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 7868535fe478faa7f98625b86eeccc35 in 436ms, sequenceid=37, compaction requested=true 2023-02-11 03:54:03,888 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 7868535fe478faa7f98625b86eeccc35: 2023-02-11 03:54:03,888 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1676087635809.7868535fe478faa7f98625b86eeccc35. 2023-02-11 03:54:03,888 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 03:54:03,888 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 03:54:03,888 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:03,889 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-02-11 03:54:03,889 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37011,1676087618428' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-02-11 03:54:03,890 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-11 03:54:03,890 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:03,891 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:03,891 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:54:03,891 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:54:03,891 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-11 03:54:03,891 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 03:54:03,891 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:54:03,891 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:54:03,891 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-11 03:54:03,892 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:03,892 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:54:03,893 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37011,1676087618428' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-02-11 03:54:03,893 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1744bad9[Count = 0] remaining members to acquire global barrier 2023-02-11 03:54:03,893 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-02-11 03:54:03,893 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-11 03:54:03,894 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-11 03:54:03,894 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-11 03:54:03,894 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-11 03:54:03,894 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:03,894 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 03:54:03,894 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-02-11 03:54:03,895 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-02-11 03:54:03,895 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,37011,1676087618428' in zk 2023-02-11 03:54:03,896 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-02-11 03:54:03,896 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:03,896 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:54:03,896 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:03,897 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:54:03,897 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:54:03,896 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-02-11 03:54:03,897 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:54:03,897 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:54:03,898 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-11 03:54:03,898 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:03,898 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:54:03,898 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-11 03:54:03,898 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:03,899 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,37011,1676087618428': 2023-02-11 03:54:03,899 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37011,1676087618428' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-02-11 03:54:03,899 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-02-11 03:54:03,899 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 03:54:03,899 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 03:54:03,899 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-02-11 03:54:03,899 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 03:54:03,902 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-11 03:54:03,902 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-11 03:54:03,902 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-11 03:54:03,902 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:54:03,902 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:54:03,902 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-11 03:54:03,902 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:03,902 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-11 03:54:03,902 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:54:03,902 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:03,902 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:03,902 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:03,903 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-11 03:54:03,903 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-11 03:54:03,903 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:54:03,903 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-11 03:54:03,903 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:03,903 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:03,904 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:54:03,904 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-11 03:54:03,904 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:03,906 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:03,906 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:03,906 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:03,906 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-02-11 03:54:03,906 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 03:54:03,906 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:03,906 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:54:03,906 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:03,906 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-11 03:54:03,906 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-11 03:54:03,906 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:03,906 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-02-11 03:54:03,907 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:03,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:03,906 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-11 03:54:03,907 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 03:54:03,907 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-11 03:54:03,907 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-11 03:54:04,007 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 03:54:04,008 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 03:54:04,013 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 03:54:04,014 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-02-11 03:54:04,015 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-02-11 03:54:04,015 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-02-11 03:54:04,015 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:54:04,015 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-02-11 03:54:04,015 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-02-11 03:54:04,016 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-11 03:54:04,016 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-11 03:54:04,017 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:04,017 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:04,017 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:04,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:04,017 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:04,017 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 03:54:04,018 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-11 03:54:04,018 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-11 03:54:04,018 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 03:54:04,018 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-11 03:54:04,018 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-02-11 03:54:04,019 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-02-11 03:54:04,019 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-02-11 03:54:04,019 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-02-11 03:54:04,019 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:54:04,020 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-02-11 03:54:04,020 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 03:54:04,020 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 03:54:04,020 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1676087635809.7868535fe478faa7f98625b86eeccc35. 2023-02-11 03:54:04,020 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1676087635809.7868535fe478faa7f98625b86eeccc35. started... 2023-02-11 03:54:04,020 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 7868535fe478faa7f98625b86eeccc35 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-11 03:54:04,436 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151120c333ef3520504e20929e1528f53dd266 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151120c333ef3520504e20929e1528f53dd266 2023-02-11 03:54:04,842 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=41, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithMonthlyPolicy/7868535fe478faa7f98625b86eeccc35/.tmp/family1/70d4f2d6d0744104b2f0ce2607a13c96 2023-02-11 03:54:04,849 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithMonthlyPolicy/7868535fe478faa7f98625b86eeccc35/.tmp/family1/70d4f2d6d0744104b2f0ce2607a13c96 as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithMonthlyPolicy/7868535fe478faa7f98625b86eeccc35/family1/70d4f2d6d0744104b2f0ce2607a13c96 2023-02-11 03:54:04,855 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithMonthlyPolicy/7868535fe478faa7f98625b86eeccc35/family1/70d4f2d6d0744104b2f0ce2607a13c96, entries=1, sequenceid=41, filesize=4.9 K 2023-02-11 03:54:04,856 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 7868535fe478faa7f98625b86eeccc35 in 836ms, sequenceid=41, compaction requested=true 2023-02-11 03:54:04,856 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 7868535fe478faa7f98625b86eeccc35: 2023-02-11 03:54:04,857 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1676087635809.7868535fe478faa7f98625b86eeccc35. 2023-02-11 03:54:04,857 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 03:54:04,857 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 03:54:04,857 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:04,857 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-02-11 03:54:04,857 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37011,1676087618428' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-02-11 03:54:04,858 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-11 03:54:04,858 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:04,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:04,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:54:04,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:54:04,859 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-11 03:54:04,859 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 03:54:04,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:54:04,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:54:04,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-11 03:54:04,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:04,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:54:04,860 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37011,1676087618428' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-02-11 03:54:04,860 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5c930137[Count = 0] remaining members to acquire global barrier 2023-02-11 03:54:04,860 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-02-11 03:54:04,860 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-11 03:54:04,862 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-11 03:54:04,863 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-11 03:54:04,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-11 03:54:04,863 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-02-11 03:54:04,863 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:04,863 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 03:54:04,863 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-02-11 03:54:04,863 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,37011,1676087618428' in zk 2023-02-11 03:54:04,864 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:04,864 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-02-11 03:54:04,865 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:04,865 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:54:04,865 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:54:04,865 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:54:04,865 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-02-11 03:54:04,866 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:54:04,866 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:54:04,866 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-11 03:54:04,866 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:04,867 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:54:04,867 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-11 03:54:04,867 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:04,868 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,37011,1676087618428': 2023-02-11 03:54:04,868 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37011,1676087618428' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-02-11 03:54:04,868 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-02-11 03:54:04,868 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 03:54:04,868 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 03:54:04,868 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-02-11 03:54:04,868 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 03:54:04,870 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-11 03:54:04,870 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-11 03:54:04,870 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-11 03:54:04,870 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-11 03:54:04,870 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:04,870 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-11 03:54:04,870 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:54:04,870 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:54:04,870 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:04,870 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:04,871 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:04,871 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-11 03:54:04,871 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:54:04,871 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-11 03:54:04,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:54:04,872 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:04,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-11 03:54:04,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:04,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:54:04,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-11 03:54:04,875 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:04,875 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-11 03:54:04,875 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:04,875 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-11 03:54:04,875 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:04,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:04,875 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:04,875 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-02-11 03:54:04,875 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 03:54:04,875 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:04,875 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-02-11 03:54:04,875 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:54:04,875 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:04,875 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-11 03:54:04,877 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-02-11 03:54:04,877 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-11 03:54:04,877 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:04,877 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-11 03:54:04,877 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 03:54:04,978 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 03:54:04,979 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 03:54:04,980 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=39921] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'}] of the table testMajorCompactionWithMonthlyPolicy 2023-02-11 03:54:04,982 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676087621935] procedure2.ProcedureExecutor(1029): Stored pid=22, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE 2023-02-11 03:54:04,983 DEBUG [PEWorker-3] locking.LockProcedure(309): LOCKED pid=22, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE 2023-02-11 03:54:04,984 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676087621935] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithMonthlyPolicy', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-11 03:54:04,984 INFO [PEWorker-3] procedure2.TimeoutExecutorThread(81): ADDED pid=22, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1676088244984 2023-02-11 03:54:04,985 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676087621935] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-02-11 03:54:04,986 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676087621935] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 10 selected files, and 0 irrelevant files table 'testMajorCompactionWithMonthlyPolicy' and column 'family1' 2023-02-11 03:54:04,986 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676087621935] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithMonthlyPolicy' column='family1' 2023-02-11 03:54:04,987 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676087621935] zookeeper.ReadOnlyZKClient(139): Connect 0x386dac4f to 127.0.0.1:53383 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-11 03:54:04,992 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676087621935] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@2b5fe029, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-11 03:54:04,992 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMajorCompactionWithMonthlyPolicy' and column='family1' 2023-02-11 03:54:05,031 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2016010395d08f83c8324f1381d54d3f3a14bd53 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e2016010395d08f83c8324f1381d54d3f3a14bd53 2023-02-11 03:54:05,032 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20160101 table='testMajorCompactionWithMonthlyPolicy' column='family1' 2023-02-11 03:54:05,048 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-11 03:54:05,049 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:35468, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-11 03:54:05,058 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-02-11 03:54:05,060 INFO [RS-EventLoopGroup-1-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:44110, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-02-11 03:54:05,072 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101/family1/9a31c77d2ec9415b9f7625bcb2644acc first=Optional[r5] last=Optional[r6] 2023-02-11 03:54:05,072 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionWithMonthlyPolicy,,1676087635809.7868535fe478faa7f98625b86eeccc35., hostname=jenkins-hbase4.apache.org,37011,1676087618428, seqNum=2 for row with hfile group [{family1,hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101/family1/9a31c77d2ec9415b9f7625bcb2644acc}] 2023-02-11 03:54:05,077 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.HStore(763): Validating hfile at hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101/family1/9a31c77d2ec9415b9f7625bcb2644acc for inclusion in 7868535fe478faa7f98625b86eeccc35/family1 2023-02-11 03:54:05,081 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.HStore(775): HFile bounds: first=r5 last=r6 2023-02-11 03:54:05,081 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.HStore(777): Region bounds: first= last= 2023-02-11 03:54:05,082 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.HRegion(2445): Flush status journal for 7868535fe478faa7f98625b86eeccc35: 2023-02-11 03:54:05,082 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101/family1/9a31c77d2ec9415b9f7625bcb2644acc to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/staging/jenkins__testMajorCompactionWithMonthlyPolicy__9rk16ju8cof6i9cdt0em5f1abv46f9bqv4i52v2l3uhps0jqk9aod7v49ru7qm75/family1/9a31c77d2ec9415b9f7625bcb2644acc 2023-02-11 03:54:05,086 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/staging/jenkins__testMajorCompactionWithMonthlyPolicy__9rk16ju8cof6i9cdt0em5f1abv46f9bqv4i52v2l3uhps0jqk9aod7v49ru7qm75/family1/9a31c77d2ec9415b9f7625bcb2644acc as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithMonthlyPolicy/7868535fe478faa7f98625b86eeccc35/family1/c53d457492884ce4b2044dd9a58c2aac_SeqId_44_ 2023-02-11 03:54:05,086 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.HStore(849): Loaded HFile hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/staging/jenkins__testMajorCompactionWithMonthlyPolicy__9rk16ju8cof6i9cdt0em5f1abv46f9bqv4i52v2l3uhps0jqk9aod7v49ru7qm75/family1/9a31c77d2ec9415b9f7625bcb2644acc into 7868535fe478faa7f98625b86eeccc35/family1 as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithMonthlyPolicy/7868535fe478faa7f98625b86eeccc35/family1/c53d457492884ce4b2044dd9a58c2aac_SeqId_44_ - updating store file list. 2023-02-11 03:54:05,092 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.HStore(882): Loaded HFile hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithMonthlyPolicy/7868535fe478faa7f98625b86eeccc35/family1/c53d457492884ce4b2044dd9a58c2aac_SeqId_44_ into 7868535fe478faa7f98625b86eeccc35/family1 2023-02-11 03:54:05,092 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.HStore(855): Successfully loaded hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/staging/jenkins__testMajorCompactionWithMonthlyPolicy__9rk16ju8cof6i9cdt0em5f1abv46f9bqv4i52v2l3uhps0jqk9aod7v49ru7qm75/family1/9a31c77d2ec9415b9f7625bcb2644acc into 7868535fe478faa7f98625b86eeccc35/family1 (new location: hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithMonthlyPolicy/7868535fe478faa7f98625b86eeccc35/family1/c53d457492884ce4b2044dd9a58c2aac_SeqId_44_) 2023-02-11 03:54:05,092 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/staging/jenkins__testMajorCompactionWithMonthlyPolicy__9rk16ju8cof6i9cdt0em5f1abv46f9bqv4i52v2l3uhps0jqk9aod7v49ru7qm75/family1/9a31c77d2ec9415b9f7625bcb2644acc 2023-02-11 03:54:05,093 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] compactions.SortedCompactionPolicy(75): Selecting compaction from 11 store files, 0 compacting, 11 eligible, 16 blocking 2023-02-11 03:54:05,098 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 10 files of size 50903 starting at candidate #1 after considering 44 permutations with 44 in ratio 2023-02-11 03:54:05,098 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.HStore(1912): 7868535fe478faa7f98625b86eeccc35/family1 is initiating minor compaction 2023-02-11 03:54:05,098 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@5cec7128; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-11 03:54:05,098 INFO [RS:0;jenkins-hbase4:37011-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 7868535fe478faa7f98625b86eeccc35/family1 in testMajorCompactionWithMonthlyPolicy,,1676087635809.7868535fe478faa7f98625b86eeccc35. 2023-02-11 03:54:05,098 INFO [RS:0;jenkins-hbase4:37011-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithMonthlyPolicy/7868535fe478faa7f98625b86eeccc35/family1/a384360e394e46bb8e08957a76677412, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithMonthlyPolicy/7868535fe478faa7f98625b86eeccc35/family1/908cdfb4203c4d70989c76de1fe2bda2, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithMonthlyPolicy/7868535fe478faa7f98625b86eeccc35/family1/935d7e82ff354caa84abfd45b607faeb, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithMonthlyPolicy/7868535fe478faa7f98625b86eeccc35/family1/885ef12989694ea09b894d857471f116, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithMonthlyPolicy/7868535fe478faa7f98625b86eeccc35/family1/821e350aedc44d50b73ec1cf0eb1706a, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithMonthlyPolicy/7868535fe478faa7f98625b86eeccc35/family1/bfe6fd6db6434ca3b905802733575006, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithMonthlyPolicy/7868535fe478faa7f98625b86eeccc35/family1/9e34002bac4847dc9e3ee80e2e09e33f, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithMonthlyPolicy/7868535fe478faa7f98625b86eeccc35/family1/9b85eda0ae454bfb821810b73c8a6124, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithMonthlyPolicy/7868535fe478faa7f98625b86eeccc35/family1/70d4f2d6d0744104b2f0ce2607a13c96, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithMonthlyPolicy/7868535fe478faa7f98625b86eeccc35/family1/c53d457492884ce4b2044dd9a58c2aac_SeqId_44_] into tmpdir=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithMonthlyPolicy/7868535fe478faa7f98625b86eeccc35/.tmp, totalSize=49.7 K 2023-02-11 03:54:05,098 DEBUG [RS:0;jenkins-hbase4:37011-shortCompactions-0] compactions.Compactor(207): Compacting a384360e394e46bb8e08957a76677412, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=9 2023-02-11 03:54:05,098 DEBUG [RS:0;jenkins-hbase4:37011-shortCompactions-0] compactions.Compactor(207): Compacting 908cdfb4203c4d70989c76de1fe2bda2, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=13 2023-02-11 03:54:05,099 DEBUG [RS:0;jenkins-hbase4:37011-shortCompactions-0] compactions.Compactor(207): Compacting 935d7e82ff354caa84abfd45b607faeb, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=17 2023-02-11 03:54:05,099 DEBUG [RS:0;jenkins-hbase4:37011-shortCompactions-0] compactions.Compactor(207): Compacting 885ef12989694ea09b894d857471f116, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=21 2023-02-11 03:54:05,099 DEBUG [RS:0;jenkins-hbase4:37011-shortCompactions-0] compactions.Compactor(207): Compacting 821e350aedc44d50b73ec1cf0eb1706a, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=25 2023-02-11 03:54:05,099 DEBUG [RS:0;jenkins-hbase4:37011-shortCompactions-0] compactions.Compactor(207): Compacting bfe6fd6db6434ca3b905802733575006, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=29 2023-02-11 03:54:05,099 DEBUG [RS:0;jenkins-hbase4:37011-shortCompactions-0] compactions.Compactor(207): Compacting 9e34002bac4847dc9e3ee80e2e09e33f, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=33 2023-02-11 03:54:05,099 DEBUG [RS:0;jenkins-hbase4:37011-shortCompactions-0] compactions.Compactor(207): Compacting 9b85eda0ae454bfb821810b73c8a6124, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=37 2023-02-11 03:54:05,099 DEBUG [RS:0;jenkins-hbase4:37011-shortCompactions-0] compactions.Compactor(207): Compacting 70d4f2d6d0744104b2f0ce2607a13c96, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=41 2023-02-11 03:54:05,099 DEBUG [RS:0;jenkins-hbase4:37011-shortCompactions-0] compactions.Compactor(207): Compacting c53d457492884ce4b2044dd9a58c2aac_SeqId_44_, keycount=2, bloomtype=ROW, size=5.2 K, encoding=NONE, compression=NONE, seqNum=45 2023-02-11 03:54:05,100 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20160101 table='testMajorCompactionWithMonthlyPolicy' column='family1' 2023-02-11 03:54:05,101 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-11 03:54:05,103 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20160101988b93ce6a8f4b70b854458640334685 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20160101988b93ce6a8f4b70b854458640334685 2023-02-11 03:54:05,105 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e201601036c675b7845d54ab39de821cb307469ea to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e201601036c675b7845d54ab39de821cb307469ea 2023-02-11 03:54:05,105 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithMonthlyPolicy 2 expired mob files are deleted 2023-02-11 03:54:05,106 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101' 2023-02-11 03:54:05,106 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 2 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMajorCompactionWithMonthlyPolicy' and column='family1' 2023-02-11 03:54:05,106 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMajorCompactionWithMonthlyPolicy' and column='family1' 2023-02-11 03:54:05,177 INFO [RS:0;jenkins-hbase4:37011-shortCompactions-0] throttle.PressureAwareThroughputController(145): 7868535fe478faa7f98625b86eeccc35#family1#compaction#22 average throughput is 0.68 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-11 03:54:05,595 DEBUG [RS:0;jenkins-hbase4:37011-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithMonthlyPolicy/7868535fe478faa7f98625b86eeccc35/.tmp/family1/61e2d18240c54a31888542ce7125d309 as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithMonthlyPolicy/7868535fe478faa7f98625b86eeccc35/family1/61e2d18240c54a31888542ce7125d309 2023-02-11 03:54:05,602 INFO [RS:0;jenkins-hbase4:37011-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 10 file(s) in 7868535fe478faa7f98625b86eeccc35/family1 of 7868535fe478faa7f98625b86eeccc35 into 61e2d18240c54a31888542ce7125d309(size=6.6 K), total size for store is 172.6 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-11 03:54:05,602 DEBUG [RS:0;jenkins-hbase4:37011-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 7868535fe478faa7f98625b86eeccc35: 2023-02-11 03:54:05,603 INFO [RS:0;jenkins-hbase4:37011-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithMonthlyPolicy,,1676087635809.7868535fe478faa7f98625b86eeccc35., storeName=family1, fileCount=10, fileSize=49.7 K (5.0 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 5.2 K), priority=2, time=1676087645098; duration=0sec 2023-02-11 03:54:05,603 DEBUG [RS:0;jenkins-hbase4:37011-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-11 03:54:05,696 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithMonthlyPolicy/7868535fe478faa7f98625b86eeccc35/family1/a384360e394e46bb8e08957a76677412, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithMonthlyPolicy/7868535fe478faa7f98625b86eeccc35/family1/908cdfb4203c4d70989c76de1fe2bda2, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithMonthlyPolicy/7868535fe478faa7f98625b86eeccc35/family1/935d7e82ff354caa84abfd45b607faeb, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithMonthlyPolicy/7868535fe478faa7f98625b86eeccc35/family1/885ef12989694ea09b894d857471f116, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithMonthlyPolicy/7868535fe478faa7f98625b86eeccc35/family1/821e350aedc44d50b73ec1cf0eb1706a, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithMonthlyPolicy/7868535fe478faa7f98625b86eeccc35/family1/bfe6fd6db6434ca3b905802733575006, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithMonthlyPolicy/7868535fe478faa7f98625b86eeccc35/family1/9e34002bac4847dc9e3ee80e2e09e33f, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithMonthlyPolicy/7868535fe478faa7f98625b86eeccc35/family1/9b85eda0ae454bfb821810b73c8a6124, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithMonthlyPolicy/7868535fe478faa7f98625b86eeccc35/family1/70d4f2d6d0744104b2f0ce2607a13c96, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithMonthlyPolicy/7868535fe478faa7f98625b86eeccc35/family1/c53d457492884ce4b2044dd9a58c2aac_SeqId_44_] to archive 2023-02-11 03:54:05,697 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-11 03:54:05,699 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithMonthlyPolicy/7868535fe478faa7f98625b86eeccc35/family1/a384360e394e46bb8e08957a76677412 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testMajorCompactionWithMonthlyPolicy/7868535fe478faa7f98625b86eeccc35/family1/a384360e394e46bb8e08957a76677412 2023-02-11 03:54:05,701 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithMonthlyPolicy/7868535fe478faa7f98625b86eeccc35/family1/908cdfb4203c4d70989c76de1fe2bda2 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testMajorCompactionWithMonthlyPolicy/7868535fe478faa7f98625b86eeccc35/family1/908cdfb4203c4d70989c76de1fe2bda2 2023-02-11 03:54:05,703 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithMonthlyPolicy/7868535fe478faa7f98625b86eeccc35/family1/935d7e82ff354caa84abfd45b607faeb to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testMajorCompactionWithMonthlyPolicy/7868535fe478faa7f98625b86eeccc35/family1/935d7e82ff354caa84abfd45b607faeb 2023-02-11 03:54:05,705 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithMonthlyPolicy/7868535fe478faa7f98625b86eeccc35/family1/885ef12989694ea09b894d857471f116 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testMajorCompactionWithMonthlyPolicy/7868535fe478faa7f98625b86eeccc35/family1/885ef12989694ea09b894d857471f116 2023-02-11 03:54:05,707 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithMonthlyPolicy/7868535fe478faa7f98625b86eeccc35/family1/821e350aedc44d50b73ec1cf0eb1706a to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testMajorCompactionWithMonthlyPolicy/7868535fe478faa7f98625b86eeccc35/family1/821e350aedc44d50b73ec1cf0eb1706a 2023-02-11 03:54:05,709 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithMonthlyPolicy/7868535fe478faa7f98625b86eeccc35/family1/bfe6fd6db6434ca3b905802733575006 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testMajorCompactionWithMonthlyPolicy/7868535fe478faa7f98625b86eeccc35/family1/bfe6fd6db6434ca3b905802733575006 2023-02-11 03:54:05,710 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithMonthlyPolicy/7868535fe478faa7f98625b86eeccc35/family1/9e34002bac4847dc9e3ee80e2e09e33f to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testMajorCompactionWithMonthlyPolicy/7868535fe478faa7f98625b86eeccc35/family1/9e34002bac4847dc9e3ee80e2e09e33f 2023-02-11 03:54:05,712 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithMonthlyPolicy/7868535fe478faa7f98625b86eeccc35/family1/9b85eda0ae454bfb821810b73c8a6124 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testMajorCompactionWithMonthlyPolicy/7868535fe478faa7f98625b86eeccc35/family1/9b85eda0ae454bfb821810b73c8a6124 2023-02-11 03:54:05,713 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithMonthlyPolicy/7868535fe478faa7f98625b86eeccc35/family1/70d4f2d6d0744104b2f0ce2607a13c96 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testMajorCompactionWithMonthlyPolicy/7868535fe478faa7f98625b86eeccc35/family1/70d4f2d6d0744104b2f0ce2607a13c96 2023-02-11 03:54:05,715 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithMonthlyPolicy/7868535fe478faa7f98625b86eeccc35/family1/c53d457492884ce4b2044dd9a58c2aac_SeqId_44_ to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testMajorCompactionWithMonthlyPolicy/7868535fe478faa7f98625b86eeccc35/family1/c53d457492884ce4b2044dd9a58c2aac_SeqId_44_ 2023-02-11 03:54:05,994 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512314c0da180c5ab46fca1c65113bcb1a47b to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e201512314c0da180c5ab46fca1c65113bcb1a47b 2023-02-11 03:54:05,995 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151201 table='testMajorCompactionWithMonthlyPolicy' column='family1' 2023-02-11 03:54:06,026 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201/family1/71604a3bf26442e6b38f58145a32bde9 first=Optional[r1] last=Optional[r4] 2023-02-11 03:54:06,027 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionWithMonthlyPolicy,,1676087635809.7868535fe478faa7f98625b86eeccc35., hostname=jenkins-hbase4.apache.org,37011,1676087618428, seqNum=2 for row with hfile group [{family1,hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201/family1/71604a3bf26442e6b38f58145a32bde9}] 2023-02-11 03:54:06,030 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.HStore(763): Validating hfile at hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201/family1/71604a3bf26442e6b38f58145a32bde9 for inclusion in 7868535fe478faa7f98625b86eeccc35/family1 2023-02-11 03:54:06,036 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.HStore(775): HFile bounds: first=r1 last=r4 2023-02-11 03:54:06,036 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.HStore(777): Region bounds: first= last= 2023-02-11 03:54:06,036 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.HRegion(2445): Flush status journal for 7868535fe478faa7f98625b86eeccc35: 2023-02-11 03:54:06,037 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201/family1/71604a3bf26442e6b38f58145a32bde9 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/staging/jenkins__testMajorCompactionWithMonthlyPolicy__aphn517g5i3e3ij5bc4s3lensmjfuhrk75ar7mistcrkaeu3qbqns4e86f7hfs1p/family1/71604a3bf26442e6b38f58145a32bde9 2023-02-11 03:54:06,040 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/staging/jenkins__testMajorCompactionWithMonthlyPolicy__aphn517g5i3e3ij5bc4s3lensmjfuhrk75ar7mistcrkaeu3qbqns4e86f7hfs1p/family1/71604a3bf26442e6b38f58145a32bde9 as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithMonthlyPolicy/7868535fe478faa7f98625b86eeccc35/family1/10ad1389cfee4ee39273edab595b4db9_SeqId_47_ 2023-02-11 03:54:06,041 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.HStore(849): Loaded HFile hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/staging/jenkins__testMajorCompactionWithMonthlyPolicy__aphn517g5i3e3ij5bc4s3lensmjfuhrk75ar7mistcrkaeu3qbqns4e86f7hfs1p/family1/71604a3bf26442e6b38f58145a32bde9 into 7868535fe478faa7f98625b86eeccc35/family1 as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithMonthlyPolicy/7868535fe478faa7f98625b86eeccc35/family1/10ad1389cfee4ee39273edab595b4db9_SeqId_47_ - updating store file list. 2023-02-11 03:54:06,048 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.HStore(882): Loaded HFile hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithMonthlyPolicy/7868535fe478faa7f98625b86eeccc35/family1/10ad1389cfee4ee39273edab595b4db9_SeqId_47_ into 7868535fe478faa7f98625b86eeccc35/family1 2023-02-11 03:54:06,048 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.HStore(855): Successfully loaded hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/staging/jenkins__testMajorCompactionWithMonthlyPolicy__aphn517g5i3e3ij5bc4s3lensmjfuhrk75ar7mistcrkaeu3qbqns4e86f7hfs1p/family1/71604a3bf26442e6b38f58145a32bde9 into 7868535fe478faa7f98625b86eeccc35/family1 (new location: hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithMonthlyPolicy/7868535fe478faa7f98625b86eeccc35/family1/10ad1389cfee4ee39273edab595b4db9_SeqId_47_) 2023-02-11 03:54:06,048 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/staging/jenkins__testMajorCompactionWithMonthlyPolicy__aphn517g5i3e3ij5bc4s3lensmjfuhrk75ar7mistcrkaeu3qbqns4e86f7hfs1p/family1/71604a3bf26442e6b38f58145a32bde9 2023-02-11 03:54:06,049 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-02-11 03:54:06,050 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 182349 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-02-11 03:54:06,050 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.HStore(1912): 7868535fe478faa7f98625b86eeccc35/family1 is initiating minor compaction (all files) 2023-02-11 03:54:06,050 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@46bebc4; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-11 03:54:06,050 INFO [RS:0;jenkins-hbase4:37011-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 7868535fe478faa7f98625b86eeccc35/family1 in testMajorCompactionWithMonthlyPolicy,,1676087635809.7868535fe478faa7f98625b86eeccc35. 2023-02-11 03:54:06,051 INFO [RS:0;jenkins-hbase4:37011-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithMonthlyPolicy/7868535fe478faa7f98625b86eeccc35/family1/90c940c0030a49f185d32ff2bc961ee5, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithMonthlyPolicy/7868535fe478faa7f98625b86eeccc35/family1/61e2d18240c54a31888542ce7125d309, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithMonthlyPolicy/7868535fe478faa7f98625b86eeccc35/family1/10ad1389cfee4ee39273edab595b4db9_SeqId_47_] into tmpdir=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithMonthlyPolicy/7868535fe478faa7f98625b86eeccc35/.tmp, totalSize=178.1 K 2023-02-11 03:54:06,051 DEBUG [RS:0;jenkins-hbase4:37011-shortCompactions-0] compactions.Compactor(207): Compacting 90c940c0030a49f185d32ff2bc961ee5, keycount=1000, bloomtype=ROW, size=166.0 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1448878842014 2023-02-11 03:54:06,052 DEBUG [RS:0;jenkins-hbase4:37011-shortCompactions-0] compactions.Compactor(207): Compacting 61e2d18240c54a31888542ce7125d309, keycount=9, bloomtype=ROW, size=6.6 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1441621242014 2023-02-11 03:54:06,053 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151201 table='testMajorCompactionWithMonthlyPolicy' column='family1' 2023-02-11 03:54:06,053 DEBUG [RS:0;jenkins-hbase4:37011-shortCompactions-0] compactions.Compactor(207): Compacting 10ad1389cfee4ee39273edab595b4db9_SeqId_47_, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=48, earliestPutTs=1448965242014 2023-02-11 03:54:06,057 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-11 03:54:06,061 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151201540ad5143d064dcea31b1b3ae061e981 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151201540ad5143d064dcea31b1b3ae061e981 2023-02-11 03:54:06,063 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151205f21209223499465fa3cc251b0c322a59 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151205f21209223499465fa3cc251b0c322a59 2023-02-11 03:54:06,065 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e201512289a05e206db7a4f46a6f1f6512b0d3bfb to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e201512289a05e206db7a4f46a6f1f6512b0d3bfb 2023-02-11 03:54:06,066 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e201512319380924a0cb94a60a2d7cd843cbedaae to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e201512319380924a0cb94a60a2d7cd843cbedaae 2023-02-11 03:54:06,066 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithMonthlyPolicy 4 expired mob files are deleted 2023-02-11 03:54:06,066 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201' 2023-02-11 03:54:06,070 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMajorCompactionWithMonthlyPolicy' and column='family1' 2023-02-11 03:54:06,071 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMajorCompactionWithMonthlyPolicy' and column='family1' 2023-02-11 03:54:06,084 INFO [RS:0;jenkins-hbase4:37011-shortCompactions-0] throttle.PressureAwareThroughputController(145): 7868535fe478faa7f98625b86eeccc35#family1#compaction#23 average throughput is 11.11 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-11 03:54:06,162 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015113003c43cf862a0494f916afaeef84cfee6 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e2015113003c43cf862a0494f916afaeef84cfee6 2023-02-11 03:54:06,163 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMajorCompactionWithMonthlyPolicy' column='family1' 2023-02-11 03:54:06,199 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101/family1/9599ea892b0c48aab71605d8d0ab5162 first=Optional[r00] last=Optional[r8] 2023-02-11 03:54:06,200 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionWithMonthlyPolicy,,1676087635809.7868535fe478faa7f98625b86eeccc35., hostname=jenkins-hbase4.apache.org,37011,1676087618428, seqNum=2 for row with hfile group [{family1,hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101/family1/9599ea892b0c48aab71605d8d0ab5162}] 2023-02-11 03:54:06,203 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.HStore(763): Validating hfile at hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101/family1/9599ea892b0c48aab71605d8d0ab5162 for inclusion in 7868535fe478faa7f98625b86eeccc35/family1 2023-02-11 03:54:06,206 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.HStore(775): HFile bounds: first=r00 last=r8 2023-02-11 03:54:06,206 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.HStore(777): Region bounds: first= last= 2023-02-11 03:54:06,207 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.HRegion(2445): Flush status journal for 7868535fe478faa7f98625b86eeccc35: 2023-02-11 03:54:06,208 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101/family1/9599ea892b0c48aab71605d8d0ab5162 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/staging/jenkins__testMajorCompactionWithMonthlyPolicy__ap5dq8hb7dr45tk5r8n5lp2vdq6icnfkjeo75993i3orpm2jajgtm0oqsgvkldhi/family1/9599ea892b0c48aab71605d8d0ab5162 2023-02-11 03:54:06,244 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-02-11 03:54:06,245 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x7228ad49 to 127.0.0.1:53383 2023-02-11 03:54:06,245 DEBUG [Finalizer] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-11 03:54:06,248 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-02-11 03:54:06,248 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x68fb9147 to 127.0.0.1:53383 2023-02-11 03:54:06,248 DEBUG [Finalizer] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-11 03:54:06,251 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/staging/jenkins__testMajorCompactionWithMonthlyPolicy__ap5dq8hb7dr45tk5r8n5lp2vdq6icnfkjeo75993i3orpm2jajgtm0oqsgvkldhi/family1/9599ea892b0c48aab71605d8d0ab5162 as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithMonthlyPolicy/7868535fe478faa7f98625b86eeccc35/family1/e2cb6cdcaa6146a08212077d171f8195_SeqId_49_ 2023-02-11 03:54:06,252 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.HStore(849): Loaded HFile hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/staging/jenkins__testMajorCompactionWithMonthlyPolicy__ap5dq8hb7dr45tk5r8n5lp2vdq6icnfkjeo75993i3orpm2jajgtm0oqsgvkldhi/family1/9599ea892b0c48aab71605d8d0ab5162 into 7868535fe478faa7f98625b86eeccc35/family1 as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithMonthlyPolicy/7868535fe478faa7f98625b86eeccc35/family1/e2cb6cdcaa6146a08212077d171f8195_SeqId_49_ - updating store file list. 2023-02-11 03:54:06,260 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.HStore(882): Loaded HFile hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithMonthlyPolicy/7868535fe478faa7f98625b86eeccc35/family1/e2cb6cdcaa6146a08212077d171f8195_SeqId_49_ into 7868535fe478faa7f98625b86eeccc35/family1 2023-02-11 03:54:06,260 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.HStore(855): Successfully loaded hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/staging/jenkins__testMajorCompactionWithMonthlyPolicy__ap5dq8hb7dr45tk5r8n5lp2vdq6icnfkjeo75993i3orpm2jajgtm0oqsgvkldhi/family1/9599ea892b0c48aab71605d8d0ab5162 into 7868535fe478faa7f98625b86eeccc35/family1 (new location: hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithMonthlyPolicy/7868535fe478faa7f98625b86eeccc35/family1/e2cb6cdcaa6146a08212077d171f8195_SeqId_49_) 2023-02-11 03:54:06,261 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/staging/jenkins__testMajorCompactionWithMonthlyPolicy__ap5dq8hb7dr45tk5r8n5lp2vdq6icnfkjeo75993i3orpm2jajgtm0oqsgvkldhi/family1/9599ea892b0c48aab71605d8d0ab5162 2023-02-11 03:54:06,264 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMajorCompactionWithMonthlyPolicy' column='family1' 2023-02-11 03:54:06,265 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-11 03:54:06,267 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151120c333ef3520504e20929e1528f53dd266 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151120c333ef3520504e20929e1528f53dd266 2023-02-11 03:54:06,269 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151128ea3aaa134ade4bf9b7e67c19e9ee20ef to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151128ea3aaa134ade4bf9b7e67c19e9ee20ef 2023-02-11 03:54:06,271 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e201511309a80a4601ae1499e81f46a5bdcbcef85 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e201511309a80a4601ae1499e81f46a5bdcbcef85 2023-02-11 03:54:06,271 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithMonthlyPolicy 3 expired mob files are deleted 2023-02-11 03:54:06,271 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101' 2023-02-11 03:54:06,271 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 3 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMajorCompactionWithMonthlyPolicy' and column='family1' 2023-02-11 03:54:06,271 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMajorCompactionWithMonthlyPolicy' and column='family1' 2023-02-11 03:54:06,271 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMajorCompactionWithMonthlyPolicy' and column='family1' 2023-02-11 03:54:06,272 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676087621935] compactions.PartitionedMobCompactor(354): After compaction, there are 4 mob files. table='testMajorCompactionWithMonthlyPolicy' column='family1' 2023-02-11 03:54:06,272 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676087621935] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithMonthlyPolicy', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-11 03:54:06,273 DEBUG [PEWorker-1] locking.LockProcedure(242): UNLOCKED pid=22, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE 2023-02-11 03:54:06,276 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=22, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE in 1.2920 sec 2023-02-11 03:54:06,487 INFO [Listener at localhost/46205] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMajorCompactionWithMonthlyPolicy Thread=251 (was 246) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:53383@0x386dac4f-EventThread sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506) Potentially hanging thread: HFileArchiver-3 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:53383@0x386dac4f-SendThread(127.0.0.1:53383) sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:345) org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1223) Potentially hanging thread: hconnection-0x37069b9a-metaLookup-shared--pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_117507140_15 at /127.0.0.1:56078 [Waiting for operation #6] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_117507140_15 at /127.0.0.1:56144 [Waiting for operation #6] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_117507140_15 at /127.0.0.1:33760 [Waiting for operation #10] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_117507140_15 at /127.0.0.1:56084 [Waiting for operation #7] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_117507140_15 at /127.0.0.1:56088 [Waiting for operation #9] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_117507140_15 at /127.0.0.1:56026 [Waiting for operation #13] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_117507140_15 at /127.0.0.1:56096 [Waiting for operation #7] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_117507140_15 at /127.0.0.1:56104 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:53383@0x386dac4f sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.DelayQueue.poll(DelayQueue.java:259) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient.run(ReadOnlyZKClient.java:324) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient$$Lambda$45/1251065594.run(Unknown Source) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_117507140_15 at /127.0.0.1:56206 [Waiting for operation #6] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_117507140_15 at /127.0.0.1:33768 [Waiting for operation #8] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_117507140_15 at /127.0.0.1:56032 [Waiting for operation #8] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x2fbe5b7a-shared-pool-23 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x37069b9a-shared-pool-3 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_117507140_15 at /127.0.0.1:55994 [Waiting for operation #12] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x37069b9a-shared-pool-5 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_117507140_15 at /127.0.0.1:33798 [Waiting for operation #11] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_117507140_15 at /127.0.0.1:56118 [Waiting for operation #14] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x37069b9a-shared-pool-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_117507140_15 at /127.0.0.1:56044 [Waiting for operation #19] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_117507140_15 at /127.0.0.1:56016 [Waiting for operation #12] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x37069b9a-shared-pool-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataStreamer for file /user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithMonthlyPolicy/7868535fe478faa7f98625b86eeccc35/.tmp/family1/7f4e2d180845416599010ebb59ba8b4c java.lang.Object.wait(Native Method) org.apache.hadoop.hdfs.DataStreamer.run(DataStreamer.java:678) Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/3712ba8e-7346-8b4d-3d93-a55f0db4c73d/cluster_d15eb7ef-e1c7-b786-c58e-de5925629f9c/dfs/data/data1/current sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1462371060_15 at /127.0.0.1:56108 [Waiting for operation #7] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1462371060_15 at /127.0.0.1:56160 [Waiting for operation #6] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x37069b9a-shared-pool-4 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x37069b9a-shared-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=594 (was 591) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=264 (was 227) - SystemLoadAverage LEAK? -, ProcessCount=168 (was 168), AvailableMemoryMB=3570 (was 3974) 2023-02-11 03:54:06,500 INFO [Listener at localhost/46205] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMinorCompactionWithWeeklyPolicy Thread=251, OpenFileDescriptor=594, MaxFileDescriptor=60000, SystemLoadAverage=264, ProcessCount=168, AvailableMemoryMB=3570 2023-02-11 03:54:06,502 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] master.HMaster$4(2102): Client=jenkins//172.31.14.131 create 'testMinorCompactionWithWeeklyPolicy', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} 2023-02-11 03:54:06,503 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] procedure2.ProcedureExecutor(1029): Stored pid=23, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:06,505 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=23, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testMinorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_PRE_OPERATION 2023-02-11 03:54:06,505 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] master.MasterRpcServices(697): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testMinorCompactionWithWeeklyPolicy" procId is: 23 2023-02-11 03:54:06,506 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=23, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testMinorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-11 03:54:06,506 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] master.MasterRpcServices(1227): Checking to see if procedure is done pid=23 2023-02-11 03:54:06,508 DEBUG [HFileArchiver-4] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/.tmp/data/default/testMinorCompactionWithWeeklyPolicy/1e125fab80f689690e5f26dc5627861b 2023-02-11 03:54:06,508 DEBUG [HFileArchiver-4] backup.HFileArchiver(153): Directory hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/.tmp/data/default/testMinorCompactionWithWeeklyPolicy/1e125fab80f689690e5f26dc5627861b empty. 2023-02-11 03:54:06,509 DEBUG [HFileArchiver-4] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/.tmp/data/default/testMinorCompactionWithWeeklyPolicy/1e125fab80f689690e5f26dc5627861b 2023-02-11 03:54:06,509 DEBUG [PEWorker-4] procedure.DeleteTableProcedure(328): Archived testMinorCompactionWithWeeklyPolicy regions 2023-02-11 03:54:06,521 DEBUG [RS:0;jenkins-hbase4:37011-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithMonthlyPolicy/7868535fe478faa7f98625b86eeccc35/.tmp/family1/7f4e2d180845416599010ebb59ba8b4c as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithMonthlyPolicy/7868535fe478faa7f98625b86eeccc35/family1/7f4e2d180845416599010ebb59ba8b4c 2023-02-11 03:54:06,521 DEBUG [PEWorker-4] util.FSTableDescriptors(570): Wrote into hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/.tmp/data/default/testMinorCompactionWithWeeklyPolicy/.tabledesc/.tableinfo.0000000001 2023-02-11 03:54:06,523 INFO [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(7671): creating {ENCODED => 1e125fab80f689690e5f26dc5627861b, NAME => 'testMinorCompactionWithWeeklyPolicy,,1676087646502.1e125fab80f689690e5f26dc5627861b.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testMinorCompactionWithWeeklyPolicy', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}, regionDir=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/.tmp 2023-02-11 03:54:06,527 INFO [RS:0;jenkins-hbase4:37011-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in 7868535fe478faa7f98625b86eeccc35/family1 of 7868535fe478faa7f98625b86eeccc35 into 7f4e2d180845416599010ebb59ba8b4c(size=167.4 K), total size for store is 333.6 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-11 03:54:06,527 DEBUG [RS:0;jenkins-hbase4:37011-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 7868535fe478faa7f98625b86eeccc35: 2023-02-11 03:54:06,527 INFO [RS:0;jenkins-hbase4:37011-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithMonthlyPolicy,,1676087635809.7868535fe478faa7f98625b86eeccc35., storeName=family1, fileCount=3, fileSize=178.1 K (166.0 K, 6.6 K, 5.5 K), priority=2, time=1676087646050; duration=0sec 2023-02-11 03:54:06,527 DEBUG [RS:0;jenkins-hbase4:37011-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-11 03:54:06,591 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithMonthlyPolicy/7868535fe478faa7f98625b86eeccc35/family1/90c940c0030a49f185d32ff2bc961ee5, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithMonthlyPolicy/7868535fe478faa7f98625b86eeccc35/family1/61e2d18240c54a31888542ce7125d309, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithMonthlyPolicy/7868535fe478faa7f98625b86eeccc35/family1/10ad1389cfee4ee39273edab595b4db9_SeqId_47_] to archive 2023-02-11 03:54:06,596 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-11 03:54:06,600 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithMonthlyPolicy/7868535fe478faa7f98625b86eeccc35/family1/90c940c0030a49f185d32ff2bc961ee5 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testMajorCompactionWithMonthlyPolicy/7868535fe478faa7f98625b86eeccc35/family1/90c940c0030a49f185d32ff2bc961ee5 2023-02-11 03:54:06,602 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithMonthlyPolicy/7868535fe478faa7f98625b86eeccc35/family1/61e2d18240c54a31888542ce7125d309 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testMajorCompactionWithMonthlyPolicy/7868535fe478faa7f98625b86eeccc35/family1/61e2d18240c54a31888542ce7125d309 2023-02-11 03:54:06,604 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithMonthlyPolicy/7868535fe478faa7f98625b86eeccc35/family1/10ad1389cfee4ee39273edab595b4db9_SeqId_47_ to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testMajorCompactionWithMonthlyPolicy/7868535fe478faa7f98625b86eeccc35/family1/10ad1389cfee4ee39273edab595b4db9_SeqId_47_ 2023-02-11 03:54:06,607 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] master.MasterRpcServices(1227): Checking to see if procedure is done pid=23 2023-02-11 03:54:06,809 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] master.MasterRpcServices(1227): Checking to see if procedure is done pid=23 2023-02-11 03:54:06,935 DEBUG [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(865): Instantiated testMinorCompactionWithWeeklyPolicy,,1676087646502.1e125fab80f689690e5f26dc5627861b.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-11 03:54:06,935 DEBUG [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1603): Closing 1e125fab80f689690e5f26dc5627861b, disabling compactions & flushes 2023-02-11 03:54:06,935 INFO [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1625): Closing region testMinorCompactionWithWeeklyPolicy,,1676087646502.1e125fab80f689690e5f26dc5627861b. 2023-02-11 03:54:06,935 DEBUG [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testMinorCompactionWithWeeklyPolicy,,1676087646502.1e125fab80f689690e5f26dc5627861b. 2023-02-11 03:54:06,935 DEBUG [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1713): Acquired close lock on testMinorCompactionWithWeeklyPolicy,,1676087646502.1e125fab80f689690e5f26dc5627861b. after waiting 0 ms 2023-02-11 03:54:06,935 DEBUG [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1723): Updates disabled for region testMinorCompactionWithWeeklyPolicy,,1676087646502.1e125fab80f689690e5f26dc5627861b. 2023-02-11 03:54:06,935 INFO [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1837): Closed testMinorCompactionWithWeeklyPolicy,,1676087646502.1e125fab80f689690e5f26dc5627861b. 2023-02-11 03:54:06,935 DEBUG [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1557): Region close journal for 1e125fab80f689690e5f26dc5627861b: Waiting for close lock at 1676087646935Disabling compacts and flushes for region at 1676087646935Disabling writes for close at 1676087646935Writing region close event to WAL at 1676087646935Closed at 1676087646935 2023-02-11 03:54:06,938 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=23, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testMinorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_ADD_TO_META 2023-02-11 03:54:06,939 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMinorCompactionWithWeeklyPolicy,,1676087646502.1e125fab80f689690e5f26dc5627861b.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1676087646939"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676087646939"}]},"ts":"1676087646939"} 2023-02-11 03:54:06,941 INFO [PEWorker-4] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-11 03:54:06,942 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=23, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testMinorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-11 03:54:06,942 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMinorCompactionWithWeeklyPolicy","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676087646942"}]},"ts":"1676087646942"} 2023-02-11 03:54:06,944 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=testMinorCompactionWithWeeklyPolicy, state=ENABLING in hbase:meta 2023-02-11 03:54:06,955 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=24, ppid=23, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMinorCompactionWithWeeklyPolicy, region=1e125fab80f689690e5f26dc5627861b, ASSIGN}] 2023-02-11 03:54:06,957 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=24, ppid=23, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMinorCompactionWithWeeklyPolicy, region=1e125fab80f689690e5f26dc5627861b, ASSIGN 2023-02-11 03:54:06,959 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=24, ppid=23, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMinorCompactionWithWeeklyPolicy, region=1e125fab80f689690e5f26dc5627861b, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,37011,1676087618428; forceNewPlan=false, retain=false 2023-02-11 03:54:07,110 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=24 updating hbase:meta row=1e125fab80f689690e5f26dc5627861b, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:07,110 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMinorCompactionWithWeeklyPolicy,,1676087646502.1e125fab80f689690e5f26dc5627861b.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1676087647110"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676087647110"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676087647110"}]},"ts":"1676087647110"} 2023-02-11 03:54:07,111 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] master.MasterRpcServices(1227): Checking to see if procedure is done pid=23 2023-02-11 03:54:07,112 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=25, ppid=24, state=RUNNABLE; OpenRegionProcedure 1e125fab80f689690e5f26dc5627861b, server=jenkins-hbase4.apache.org,37011,1676087618428}] 2023-02-11 03:54:07,268 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMinorCompactionWithWeeklyPolicy,,1676087646502.1e125fab80f689690e5f26dc5627861b. 2023-02-11 03:54:07,268 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 1e125fab80f689690e5f26dc5627861b, NAME => 'testMinorCompactionWithWeeklyPolicy,,1676087646502.1e125fab80f689690e5f26dc5627861b.', STARTKEY => '', ENDKEY => ''} 2023-02-11 03:54:07,269 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMinorCompactionWithWeeklyPolicy 1e125fab80f689690e5f26dc5627861b 2023-02-11 03:54:07,269 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testMinorCompactionWithWeeklyPolicy,,1676087646502.1e125fab80f689690e5f26dc5627861b.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-11 03:54:07,269 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 1e125fab80f689690e5f26dc5627861b 2023-02-11 03:54:07,269 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 1e125fab80f689690e5f26dc5627861b 2023-02-11 03:54:07,270 INFO [StoreOpener-1e125fab80f689690e5f26dc5627861b-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 1e125fab80f689690e5f26dc5627861b 2023-02-11 03:54:07,272 DEBUG [StoreOpener-1e125fab80f689690e5f26dc5627861b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMinorCompactionWithWeeklyPolicy/1e125fab80f689690e5f26dc5627861b/family1 2023-02-11 03:54:07,272 DEBUG [StoreOpener-1e125fab80f689690e5f26dc5627861b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMinorCompactionWithWeeklyPolicy/1e125fab80f689690e5f26dc5627861b/family1 2023-02-11 03:54:07,272 INFO [StoreOpener-1e125fab80f689690e5f26dc5627861b-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1e125fab80f689690e5f26dc5627861b columnFamilyName family1 2023-02-11 03:54:07,274 INFO [StoreOpener-1e125fab80f689690e5f26dc5627861b-1] regionserver.HStore(310): Store=1e125fab80f689690e5f26dc5627861b/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 03:54:07,275 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMinorCompactionWithWeeklyPolicy/1e125fab80f689690e5f26dc5627861b 2023-02-11 03:54:07,275 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMinorCompactionWithWeeklyPolicy/1e125fab80f689690e5f26dc5627861b 2023-02-11 03:54:07,278 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 1e125fab80f689690e5f26dc5627861b 2023-02-11 03:54:07,280 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMinorCompactionWithWeeklyPolicy/1e125fab80f689690e5f26dc5627861b/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-11 03:54:07,281 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 1e125fab80f689690e5f26dc5627861b; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=62169406, jitterRate=-0.07360365986824036}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-11 03:54:07,281 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 1e125fab80f689690e5f26dc5627861b: Running coprocessor pre-open hook at 1676087647269Writing region info on filesystem at 1676087647269Initializing all the Stores at 1676087647270 (+1 ms)Instantiating store for column family {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} at 1676087647270Cleaning up temporary data from old regions at 1676087647275 (+5 ms)Cleaning up detritus from prior splits at 1676087647276 (+1 ms)Running coprocessor post-open hooks at 1676087647281 (+5 ms)Region opened successfully at 1676087647281 2023-02-11 03:54:07,282 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMinorCompactionWithWeeklyPolicy,,1676087646502.1e125fab80f689690e5f26dc5627861b., pid=25, masterSystemTime=1676087647264 2023-02-11 03:54:07,284 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMinorCompactionWithWeeklyPolicy,,1676087646502.1e125fab80f689690e5f26dc5627861b. 2023-02-11 03:54:07,284 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMinorCompactionWithWeeklyPolicy,,1676087646502.1e125fab80f689690e5f26dc5627861b. 2023-02-11 03:54:07,284 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=24 updating hbase:meta row=1e125fab80f689690e5f26dc5627861b, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:07,284 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMinorCompactionWithWeeklyPolicy,,1676087646502.1e125fab80f689690e5f26dc5627861b.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1676087647284"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676087647284"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676087647284"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676087647284"}]},"ts":"1676087647284"} 2023-02-11 03:54:07,288 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=25, resume processing ppid=24 2023-02-11 03:54:07,288 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=25, ppid=24, state=SUCCESS; OpenRegionProcedure 1e125fab80f689690e5f26dc5627861b, server=jenkins-hbase4.apache.org,37011,1676087618428 in 174 msec 2023-02-11 03:54:07,290 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=24, resume processing ppid=23 2023-02-11 03:54:07,290 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=24, ppid=23, state=SUCCESS; TransitRegionStateProcedure table=testMinorCompactionWithWeeklyPolicy, region=1e125fab80f689690e5f26dc5627861b, ASSIGN in 333 msec 2023-02-11 03:54:07,291 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=23, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testMinorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-11 03:54:07,291 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMinorCompactionWithWeeklyPolicy","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676087647291"}]},"ts":"1676087647291"} 2023-02-11 03:54:07,292 INFO [PEWorker-5] hbase.MetaTableAccessor(1635): Updated tableName=testMinorCompactionWithWeeklyPolicy, state=ENABLED in hbase:meta 2023-02-11 03:54:07,295 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=23, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testMinorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_POST_OPERATION 2023-02-11 03:54:07,296 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=23, state=SUCCESS; CreateTableProcedure table=testMinorCompactionWithWeeklyPolicy in 793 msec 2023-02-11 03:54:07,612 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] master.MasterRpcServices(1227): Checking to see if procedure is done pid=23 2023-02-11 03:54:07,612 INFO [Listener at localhost/46205] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testMinorCompactionWithWeeklyPolicy, procId: 23 completed 2023-02-11 03:54:07,636 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 03:54:07,637 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:07,638 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-02-11 03:54:07,638 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:54:07,638 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-02-11 03:54:07,638 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-02-11 03:54:07,638 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:07,638 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:07,639 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:07,639 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:07,639 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:07,640 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:07,640 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:07,640 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 03:54:07,640 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:07,640 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:07,641 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 03:54:07,641 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:07,641 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:07,641 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:07,641 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-02-11 03:54:07,641 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:54:07,641 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-02-11 03:54:07,641 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 03:54:07,641 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 03:54:07,641 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1676087646502.1e125fab80f689690e5f26dc5627861b. 2023-02-11 03:54:07,642 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1676087646502.1e125fab80f689690e5f26dc5627861b. started... 2023-02-11 03:54:07,642 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 1e125fab80f689690e5f26dc5627861b 1/1 column families, dataSize=74.11 KB heapSize=148.69 KB 2023-02-11 03:54:08,062 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151130c3c57b3593544cffa45b03cd0c699619 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20151130c3c57b3593544cffa45b03cd0c699619 2023-02-11 03:54:08,466 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=74.1 K, hasBloomFilter=true, into tmp file hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMinorCompactionWithWeeklyPolicy/1e125fab80f689690e5f26dc5627861b/.tmp/family1/fc01e6adad6841f89baac037109ec002 2023-02-11 03:54:08,472 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMinorCompactionWithWeeklyPolicy/1e125fab80f689690e5f26dc5627861b/.tmp/family1/fc01e6adad6841f89baac037109ec002 as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMinorCompactionWithWeeklyPolicy/1e125fab80f689690e5f26dc5627861b/family1/fc01e6adad6841f89baac037109ec002 2023-02-11 03:54:08,477 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMinorCompactionWithWeeklyPolicy/1e125fab80f689690e5f26dc5627861b/family1/fc01e6adad6841f89baac037109ec002, entries=1000, sequenceid=5, filesize=165.1 K 2023-02-11 03:54:08,478 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~74.11 KB/75890, heapSize ~148.67 KB/152240, currentSize=0 B/0 for 1e125fab80f689690e5f26dc5627861b in 836ms, sequenceid=5, compaction requested=false 2023-02-11 03:54:08,478 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testMinorCompactionWithWeeklyPolicy' 2023-02-11 03:54:08,478 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 1e125fab80f689690e5f26dc5627861b: 2023-02-11 03:54:08,478 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1676087646502.1e125fab80f689690e5f26dc5627861b. 2023-02-11 03:54:08,478 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 03:54:08,478 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 03:54:08,478 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:08,479 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-02-11 03:54:08,479 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37011,1676087618428' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-02-11 03:54:08,480 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:08,480 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:08,480 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:08,480 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:54:08,480 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:54:08,480 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:08,480 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 03:54:08,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:54:08,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:54:08,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:08,482 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:08,482 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:54:08,482 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37011,1676087618428' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-02-11 03:54:08,482 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1bd93fec[Count = 0] remaining members to acquire global barrier 2023-02-11 03:54:08,482 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-02-11 03:54:08,482 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:08,484 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:08,484 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:08,484 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:08,484 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-02-11 03:54:08,484 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-02-11 03:54:08,484 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,37011,1676087618428' in zk 2023-02-11 03:54:08,484 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:08,484 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 03:54:08,487 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:08,487 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-02-11 03:54:08,487 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:08,487 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:54:08,487 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:54:08,487 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:54:08,487 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-02-11 03:54:08,487 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:54:08,488 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:54:08,488 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:08,488 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:08,488 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:54:08,489 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:08,489 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:08,489 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,37011,1676087618428': 2023-02-11 03:54:08,489 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37011,1676087618428' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-02-11 03:54:08,489 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-02-11 03:54:08,489 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 03:54:08,489 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 03:54:08,490 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:08,490 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 03:54:08,491 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:08,491 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:08,491 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:08,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:08,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:54:08,492 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:54:08,492 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:08,492 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:08,492 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:08,492 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:54:08,492 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:08,492 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:08,493 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:08,493 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:08,493 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:54:08,493 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:08,494 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:08,494 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:08,494 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:54:08,494 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:08,495 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:08,496 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:08,496 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:08,496 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:08,496 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:08,496 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-02-11 03:54:08,496 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:08,496 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-02-11 03:54:08,496 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 03:54:08,496 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:54:08,496 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:08,496 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:08,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:08,497 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:08,497 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-02-11 03:54:08,497 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 03:54:08,497 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:08,497 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:08,497 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:08,497 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:08,597 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 03:54:08,598 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 03:54:08,603 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 03:54:08,604 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-02-11 03:54:08,604 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:08,604 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-02-11 03:54:08,604 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:54:08,604 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-02-11 03:54:08,604 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-02-11 03:54:08,605 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:08,605 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:08,606 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:08,606 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:08,606 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:08,606 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:08,607 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:08,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:08,607 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 03:54:08,607 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:08,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 03:54:08,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:08,607 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:08,608 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-02-11 03:54:08,608 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:08,608 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-02-11 03:54:08,608 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:54:08,608 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-02-11 03:54:08,608 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 03:54:08,608 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 03:54:08,608 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1676087646502.1e125fab80f689690e5f26dc5627861b. 2023-02-11 03:54:08,608 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1676087646502.1e125fab80f689690e5f26dc5627861b. started... 2023-02-11 03:54:08,608 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 1e125fab80f689690e5f26dc5627861b 1/1 column families, dataSize=74 B heapSize=408 B 2023-02-11 03:54:08,623 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201511282d2b18b5b9d84b70864a7d1506f6aa29 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e201511282d2b18b5b9d84b70864a7d1506f6aa29 2023-02-11 03:54:09,030 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=74, hasBloomFilter=true, into tmp file hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMinorCompactionWithWeeklyPolicy/1e125fab80f689690e5f26dc5627861b/.tmp/family1/1b6f4852f6e34577a74e2649ed4aa1d5 2023-02-11 03:54:09,035 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMinorCompactionWithWeeklyPolicy/1e125fab80f689690e5f26dc5627861b/.tmp/family1/1b6f4852f6e34577a74e2649ed4aa1d5 as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMinorCompactionWithWeeklyPolicy/1e125fab80f689690e5f26dc5627861b/family1/1b6f4852f6e34577a74e2649ed4aa1d5 2023-02-11 03:54:09,040 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMinorCompactionWithWeeklyPolicy/1e125fab80f689690e5f26dc5627861b/family1/1b6f4852f6e34577a74e2649ed4aa1d5, entries=1, sequenceid=9, filesize=5.0 K 2023-02-11 03:54:09,041 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~74 B/74, heapSize ~392 B/392, currentSize=0 B/0 for 1e125fab80f689690e5f26dc5627861b in 433ms, sequenceid=9, compaction requested=false 2023-02-11 03:54:09,042 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 1e125fab80f689690e5f26dc5627861b: 2023-02-11 03:54:09,042 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1676087646502.1e125fab80f689690e5f26dc5627861b. 2023-02-11 03:54:09,042 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 03:54:09,042 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 03:54:09,042 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:09,042 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-02-11 03:54:09,042 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37011,1676087618428' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-02-11 03:54:09,049 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:09,049 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:09,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:09,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:54:09,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:54:09,050 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:09,050 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 03:54:09,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:54:09,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:54:09,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:09,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:09,051 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:54:09,051 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37011,1676087618428' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-02-11 03:54:09,051 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5574fc0c[Count = 0] remaining members to acquire global barrier 2023-02-11 03:54:09,051 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-02-11 03:54:09,051 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:09,053 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:09,053 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:09,053 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:09,053 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-02-11 03:54:09,053 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-02-11 03:54:09,053 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,37011,1676087618428' in zk 2023-02-11 03:54:09,053 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:09,053 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 03:54:09,054 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:09,054 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-02-11 03:54:09,054 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:09,054 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:54:09,054 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:54:09,054 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:54:09,054 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-02-11 03:54:09,054 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:54:09,055 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:54:09,055 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:09,055 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:09,055 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:54:09,055 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:09,056 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:09,056 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,37011,1676087618428': 2023-02-11 03:54:09,056 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37011,1676087618428' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-02-11 03:54:09,056 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-02-11 03:54:09,057 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 03:54:09,057 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 03:54:09,057 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:09,057 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 03:54:09,058 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:09,058 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:09,058 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:09,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:09,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:54:09,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:54:09,058 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:09,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:09,059 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:54:09,059 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:09,059 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:09,059 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:09,059 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:09,059 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:09,059 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:54:09,060 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:09,060 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:09,060 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:09,060 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:54:09,060 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:09,061 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:09,064 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:09,064 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:09,064 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:09,064 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:09,064 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-02-11 03:54:09,065 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 03:54:09,064 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:09,064 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:54:09,065 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:09,065 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:09,065 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:09,065 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:09,065 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:09,065 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-02-11 03:54:09,065 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:09,065 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 03:54:09,065 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:09,065 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:09,165 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 03:54:09,166 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 03:54:09,171 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 03:54:09,173 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-02-11 03:54:09,173 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:09,173 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-02-11 03:54:09,173 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:54:09,173 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-02-11 03:54:09,173 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-02-11 03:54:09,173 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:09,173 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:09,175 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:09,175 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:09,175 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:09,175 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:09,175 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:09,175 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 03:54:09,175 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:09,176 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:09,176 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 03:54:09,176 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:09,176 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:09,177 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-02-11 03:54:09,177 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:09,177 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-02-11 03:54:09,177 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:54:09,177 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-02-11 03:54:09,178 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 03:54:09,178 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 03:54:09,178 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1676087646502.1e125fab80f689690e5f26dc5627861b. 2023-02-11 03:54:09,178 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1676087646502.1e125fab80f689690e5f26dc5627861b. started... 2023-02-11 03:54:09,178 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 1e125fab80f689690e5f26dc5627861b 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-11 03:54:09,194 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151201afe758114a764bcaa3aa3ae8d9864cd7 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20151201afe758114a764bcaa3aa3ae8d9864cd7 2023-02-11 03:54:09,600 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMinorCompactionWithWeeklyPolicy/1e125fab80f689690e5f26dc5627861b/.tmp/family1/08ff73d09b584551af0e99a28732cf57 2023-02-11 03:54:09,608 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMinorCompactionWithWeeklyPolicy/1e125fab80f689690e5f26dc5627861b/.tmp/family1/08ff73d09b584551af0e99a28732cf57 as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMinorCompactionWithWeeklyPolicy/1e125fab80f689690e5f26dc5627861b/family1/08ff73d09b584551af0e99a28732cf57 2023-02-11 03:54:09,613 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMinorCompactionWithWeeklyPolicy/1e125fab80f689690e5f26dc5627861b/family1/08ff73d09b584551af0e99a28732cf57, entries=1, sequenceid=13, filesize=4.9 K 2023-02-11 03:54:09,614 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 1e125fab80f689690e5f26dc5627861b in 436ms, sequenceid=13, compaction requested=true 2023-02-11 03:54:09,614 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 1e125fab80f689690e5f26dc5627861b: 2023-02-11 03:54:09,614 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1676087646502.1e125fab80f689690e5f26dc5627861b. 2023-02-11 03:54:09,614 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 03:54:09,614 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 03:54:09,614 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:09,614 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-02-11 03:54:09,614 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37011,1676087618428' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-02-11 03:54:09,616 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:09,616 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:09,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:09,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:54:09,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:54:09,616 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:09,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:54:09,616 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 03:54:09,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:54:09,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:09,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:09,618 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:54:09,618 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37011,1676087618428' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-02-11 03:54:09,618 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@67d40609[Count = 0] remaining members to acquire global barrier 2023-02-11 03:54:09,618 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-02-11 03:54:09,618 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:09,619 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:09,619 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:09,619 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:09,619 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-02-11 03:54:09,620 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-02-11 03:54:09,620 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,37011,1676087618428' in zk 2023-02-11 03:54:09,620 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:09,620 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 03:54:09,622 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-02-11 03:54:09,622 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:09,622 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:54:09,622 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:09,622 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:54:09,622 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:54:09,622 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-02-11 03:54:09,622 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:54:09,622 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:54:09,623 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:09,623 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:09,623 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:54:09,624 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:09,624 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:09,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,37011,1676087618428': 2023-02-11 03:54:09,625 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37011,1676087618428' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-02-11 03:54:09,625 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-02-11 03:54:09,625 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 03:54:09,625 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 03:54:09,625 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:09,625 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 03:54:09,626 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:09,627 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:09,627 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:09,627 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:09,627 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:54:09,627 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:54:09,627 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:09,627 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:09,627 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:54:09,627 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:09,627 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:09,627 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:09,627 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:09,628 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:09,628 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:54:09,628 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:09,628 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:09,629 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:09,629 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:54:09,629 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:09,629 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:09,633 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:09,633 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:09,633 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:09,634 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:09,634 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:54:09,634 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:09,634 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-02-11 03:54:09,634 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 03:54:09,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:09,634 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:09,634 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:09,634 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:09,634 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:09,634 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:09,634 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-02-11 03:54:09,634 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:09,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:09,634 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 03:54:09,735 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 03:54:09,736 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 03:54:09,740 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 03:54:09,742 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-02-11 03:54:09,742 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:09,742 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-02-11 03:54:09,742 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:54:09,742 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-02-11 03:54:09,742 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-02-11 03:54:09,742 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:09,742 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:09,745 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:09,745 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:09,745 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:09,745 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:09,745 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:09,745 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 03:54:09,745 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:09,745 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:09,746 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 03:54:09,746 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:09,746 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:09,746 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-02-11 03:54:09,746 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:09,746 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-02-11 03:54:09,746 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:54:09,746 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-02-11 03:54:09,746 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 03:54:09,746 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 03:54:09,746 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1676087646502.1e125fab80f689690e5f26dc5627861b. 2023-02-11 03:54:09,746 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1676087646502.1e125fab80f689690e5f26dc5627861b. started... 2023-02-11 03:54:09,746 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 1e125fab80f689690e5f26dc5627861b 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-11 03:54:10,160 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512054878b734d81944dc8f0f0dc689e0906d to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e201512054878b734d81944dc8f0f0dc689e0906d 2023-02-11 03:54:10,566 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMinorCompactionWithWeeklyPolicy/1e125fab80f689690e5f26dc5627861b/.tmp/family1/66cd78939a4748e0b1c48b19ebc2b58c 2023-02-11 03:54:10,572 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMinorCompactionWithWeeklyPolicy/1e125fab80f689690e5f26dc5627861b/.tmp/family1/66cd78939a4748e0b1c48b19ebc2b58c as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMinorCompactionWithWeeklyPolicy/1e125fab80f689690e5f26dc5627861b/family1/66cd78939a4748e0b1c48b19ebc2b58c 2023-02-11 03:54:10,576 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMinorCompactionWithWeeklyPolicy/1e125fab80f689690e5f26dc5627861b/family1/66cd78939a4748e0b1c48b19ebc2b58c, entries=1, sequenceid=17, filesize=4.9 K 2023-02-11 03:54:10,577 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 1e125fab80f689690e5f26dc5627861b in 831ms, sequenceid=17, compaction requested=true 2023-02-11 03:54:10,577 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 1e125fab80f689690e5f26dc5627861b: 2023-02-11 03:54:10,578 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1676087646502.1e125fab80f689690e5f26dc5627861b. 2023-02-11 03:54:10,578 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 03:54:10,578 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 03:54:10,578 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:10,578 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-02-11 03:54:10,578 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37011,1676087618428' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-02-11 03:54:10,579 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:10,579 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:10,579 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:10,579 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:54:10,579 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:54:10,580 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:54:10,580 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:10,580 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 03:54:10,580 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:54:10,580 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:10,580 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:10,581 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:54:10,581 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37011,1676087618428' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-02-11 03:54:10,581 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@238a397d[Count = 0] remaining members to acquire global barrier 2023-02-11 03:54:10,581 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-02-11 03:54:10,581 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:10,582 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:10,582 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:10,582 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:10,582 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-02-11 03:54:10,582 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-02-11 03:54:10,582 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,37011,1676087618428' in zk 2023-02-11 03:54:10,582 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:10,582 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 03:54:10,584 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-02-11 03:54:10,584 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:10,584 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:54:10,584 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:10,584 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:54:10,584 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:54:10,584 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-02-11 03:54:10,584 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:54:10,585 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:54:10,585 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:10,585 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:10,585 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:54:10,585 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:10,586 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:10,586 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,37011,1676087618428': 2023-02-11 03:54:10,586 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37011,1676087618428' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-02-11 03:54:10,586 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-02-11 03:54:10,586 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 03:54:10,586 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 03:54:10,586 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:10,586 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 03:54:10,589 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:10,589 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:10,589 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:10,589 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:10,589 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:10,589 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:54:10,589 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:54:10,589 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:10,589 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:10,589 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:10,589 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:54:10,589 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:10,589 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:10,589 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:10,590 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:54:10,590 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:10,591 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:10,591 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:10,591 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:54:10,591 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:10,591 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:10,593 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:10,593 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:10,593 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:10,593 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-02-11 03:54:10,593 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 03:54:10,593 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-02-11 03:54:10,593 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:10,593 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:10,593 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:10,593 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:54:10,593 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-02-11 03:54:10,593 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:10,593 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 03:54:10,593 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:10,593 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:10,593 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:10,594 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:10,594 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:10,594 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:10,693 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 03:54:10,694 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 03:54:10,699 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 03:54:10,700 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-02-11 03:54:10,700 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:10,700 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-02-11 03:54:10,700 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:54:10,701 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-02-11 03:54:10,701 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-02-11 03:54:10,701 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:10,701 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:10,702 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:10,702 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:10,702 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:10,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:10,702 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:10,702 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 03:54:10,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:10,703 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:10,703 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 03:54:10,703 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:10,703 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:10,703 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-02-11 03:54:10,703 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:10,703 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-02-11 03:54:10,703 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:54:10,703 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-02-11 03:54:10,704 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 03:54:10,704 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 03:54:10,704 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1676087646502.1e125fab80f689690e5f26dc5627861b. 2023-02-11 03:54:10,704 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1676087646502.1e125fab80f689690e5f26dc5627861b. started... 2023-02-11 03:54:10,704 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 1e125fab80f689690e5f26dc5627861b 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-11 03:54:11,118 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015122863a2b2fb23a7458c90fb0f5790ed42a9 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e2015122863a2b2fb23a7458c90fb0f5790ed42a9 2023-02-11 03:54:11,524 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=21, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMinorCompactionWithWeeklyPolicy/1e125fab80f689690e5f26dc5627861b/.tmp/family1/5808fd18d56e42c1ba627a0422c00010 2023-02-11 03:54:11,532 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMinorCompactionWithWeeklyPolicy/1e125fab80f689690e5f26dc5627861b/.tmp/family1/5808fd18d56e42c1ba627a0422c00010 as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMinorCompactionWithWeeklyPolicy/1e125fab80f689690e5f26dc5627861b/family1/5808fd18d56e42c1ba627a0422c00010 2023-02-11 03:54:11,538 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMinorCompactionWithWeeklyPolicy/1e125fab80f689690e5f26dc5627861b/family1/5808fd18d56e42c1ba627a0422c00010, entries=1, sequenceid=21, filesize=4.9 K 2023-02-11 03:54:11,540 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 1e125fab80f689690e5f26dc5627861b in 836ms, sequenceid=21, compaction requested=true 2023-02-11 03:54:11,540 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 1e125fab80f689690e5f26dc5627861b: 2023-02-11 03:54:11,540 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1676087646502.1e125fab80f689690e5f26dc5627861b. 2023-02-11 03:54:11,540 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 03:54:11,540 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 03:54:11,540 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:11,540 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-02-11 03:54:11,540 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37011,1676087618428' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-02-11 03:54:11,542 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:11,542 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:11,542 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:11,542 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:54:11,542 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:54:11,543 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:11,543 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 03:54:11,543 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:54:11,543 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:54:11,544 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:11,544 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:11,545 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:54:11,545 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37011,1676087618428' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-02-11 03:54:11,545 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@717e44bf[Count = 0] remaining members to acquire global barrier 2023-02-11 03:54:11,545 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-02-11 03:54:11,545 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:11,548 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:11,548 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:11,548 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:11,548 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-02-11 03:54:11,548 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:11,548 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 03:54:11,548 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-02-11 03:54:11,548 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,37011,1676087618428' in zk 2023-02-11 03:54:11,550 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-02-11 03:54:11,550 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:11,550 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:54:11,550 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:11,550 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:54:11,550 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:54:11,550 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-02-11 03:54:11,550 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:54:11,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:54:11,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:11,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:11,552 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:54:11,553 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:11,553 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:11,553 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,37011,1676087618428': 2023-02-11 03:54:11,554 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37011,1676087618428' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-02-11 03:54:11,554 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-02-11 03:54:11,554 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 03:54:11,554 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 03:54:11,554 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:11,554 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 03:54:11,555 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:11,555 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:11,556 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:11,556 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:11,556 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:54:11,556 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:54:11,556 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:11,556 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:11,556 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:54:11,556 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:11,556 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:11,557 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:11,557 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:11,557 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:11,558 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:54:11,558 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:11,558 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:11,559 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:11,559 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:54:11,560 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:11,561 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:11,562 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:11,562 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:11,562 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:11,562 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:11,562 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:11,562 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:11,562 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:11,562 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:11,563 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:11,562 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:11,563 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:11,563 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:54:11,563 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-02-11 03:54:11,563 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 03:54:11,563 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:11,563 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-02-11 03:54:11,563 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 03:54:11,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:11,565 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-02-11 03:54:11,664 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 03:54:11,665 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 03:54:11,671 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 03:54:11,672 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-02-11 03:54:11,672 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:11,672 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-02-11 03:54:11,673 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:54:11,673 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-02-11 03:54:11,673 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-02-11 03:54:11,673 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:11,673 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:11,676 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:11,676 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:11,676 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:11,676 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:11,676 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:11,676 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:11,676 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 03:54:11,677 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:11,677 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 03:54:11,677 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:11,677 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:11,680 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-02-11 03:54:11,680 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:11,680 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-02-11 03:54:11,680 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:54:11,680 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-02-11 03:54:11,681 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 03:54:11,681 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 03:54:11,681 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1676087646502.1e125fab80f689690e5f26dc5627861b. 2023-02-11 03:54:11,681 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1676087646502.1e125fab80f689690e5f26dc5627861b. started... 2023-02-11 03:54:11,682 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 1e125fab80f689690e5f26dc5627861b 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-11 03:54:11,699 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151231d286abba7c0b49668947e555bc48bc68 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20151231d286abba7c0b49668947e555bc48bc68 2023-02-11 03:54:12,106 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=25, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMinorCompactionWithWeeklyPolicy/1e125fab80f689690e5f26dc5627861b/.tmp/family1/108050f1493a4b4cac8260464fc349cc 2023-02-11 03:54:12,111 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMinorCompactionWithWeeklyPolicy/1e125fab80f689690e5f26dc5627861b/.tmp/family1/108050f1493a4b4cac8260464fc349cc as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMinorCompactionWithWeeklyPolicy/1e125fab80f689690e5f26dc5627861b/family1/108050f1493a4b4cac8260464fc349cc 2023-02-11 03:54:12,116 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMinorCompactionWithWeeklyPolicy/1e125fab80f689690e5f26dc5627861b/family1/108050f1493a4b4cac8260464fc349cc, entries=1, sequenceid=25, filesize=4.9 K 2023-02-11 03:54:12,117 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 1e125fab80f689690e5f26dc5627861b in 435ms, sequenceid=25, compaction requested=true 2023-02-11 03:54:12,117 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 1e125fab80f689690e5f26dc5627861b: 2023-02-11 03:54:12,117 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1676087646502.1e125fab80f689690e5f26dc5627861b. 2023-02-11 03:54:12,117 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 03:54:12,117 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 03:54:12,117 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:12,117 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-02-11 03:54:12,117 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37011,1676087618428' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-02-11 03:54:12,119 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:12,119 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:12,119 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:12,119 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:54:12,119 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:54:12,119 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:12,119 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 03:54:12,119 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:54:12,120 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:54:12,120 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:12,120 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:12,120 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:54:12,121 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37011,1676087618428' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-02-11 03:54:12,121 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3985200[Count = 0] remaining members to acquire global barrier 2023-02-11 03:54:12,121 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-02-11 03:54:12,121 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:12,122 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:12,122 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:12,122 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:12,122 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-02-11 03:54:12,122 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-02-11 03:54:12,122 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,37011,1676087618428' in zk 2023-02-11 03:54:12,122 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:12,123 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 03:54:12,127 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-02-11 03:54:12,127 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:12,127 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:54:12,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:12,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:54:12,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:54:12,127 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-02-11 03:54:12,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:54:12,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:54:12,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:12,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:12,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:54:12,129 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:12,129 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:12,129 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,37011,1676087618428': 2023-02-11 03:54:12,130 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37011,1676087618428' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-02-11 03:54:12,130 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-02-11 03:54:12,130 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 03:54:12,130 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 03:54:12,130 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:12,130 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 03:54:12,131 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:12,131 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:12,131 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:12,131 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:54:12,131 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:54:12,131 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:12,131 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:12,131 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:12,132 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:54:12,132 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:12,132 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:12,132 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:12,132 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:12,132 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:12,132 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:54:12,132 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:12,133 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:12,133 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:12,133 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:54:12,133 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:12,134 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:12,135 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:12,135 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:12,135 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:12,135 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:12,135 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:12,135 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:12,135 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:54:12,135 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:12,135 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-02-11 03:54:12,135 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 03:54:12,135 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:12,135 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-02-11 03:54:12,135 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:12,135 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:12,135 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:12,135 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-02-11 03:54:12,135 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 03:54:12,136 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:12,136 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:12,236 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 03:54:12,237 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 03:54:12,241 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 03:54:12,243 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-02-11 03:54:12,243 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:12,243 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-02-11 03:54:12,243 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:54:12,243 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-02-11 03:54:12,243 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-02-11 03:54:12,243 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:12,244 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:12,245 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:12,245 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:12,245 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:12,245 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:12,246 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:12,246 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 03:54:12,246 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:12,246 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:12,247 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 03:54:12,247 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:12,247 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:12,247 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-02-11 03:54:12,247 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:12,247 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-02-11 03:54:12,247 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:54:12,247 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-02-11 03:54:12,247 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 03:54:12,247 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 03:54:12,247 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1676087646502.1e125fab80f689690e5f26dc5627861b. 2023-02-11 03:54:12,247 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1676087646502.1e125fab80f689690e5f26dc5627861b. started... 2023-02-11 03:54:12,247 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 1e125fab80f689690e5f26dc5627861b 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-11 03:54:12,263 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201601012ac79880d52a4799a9477fda0da5c2ea to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e201601012ac79880d52a4799a9477fda0da5c2ea 2023-02-11 03:54:12,277 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=29, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMinorCompactionWithWeeklyPolicy/1e125fab80f689690e5f26dc5627861b/.tmp/family1/cd4a04303ec2472c859d660fd8b89609 2023-02-11 03:54:12,283 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMinorCompactionWithWeeklyPolicy/1e125fab80f689690e5f26dc5627861b/.tmp/family1/cd4a04303ec2472c859d660fd8b89609 as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMinorCompactionWithWeeklyPolicy/1e125fab80f689690e5f26dc5627861b/family1/cd4a04303ec2472c859d660fd8b89609 2023-02-11 03:54:12,290 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMinorCompactionWithWeeklyPolicy/1e125fab80f689690e5f26dc5627861b/family1/cd4a04303ec2472c859d660fd8b89609, entries=1, sequenceid=29, filesize=4.9 K 2023-02-11 03:54:12,291 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 1e125fab80f689690e5f26dc5627861b in 44ms, sequenceid=29, compaction requested=true 2023-02-11 03:54:12,291 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 1e125fab80f689690e5f26dc5627861b: 2023-02-11 03:54:12,291 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1676087646502.1e125fab80f689690e5f26dc5627861b. 2023-02-11 03:54:12,291 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 03:54:12,291 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 03:54:12,291 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:12,292 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-02-11 03:54:12,292 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37011,1676087618428' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-02-11 03:54:12,295 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:12,295 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:12,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:12,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:54:12,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:54:12,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:54:12,296 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:12,296 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 03:54:12,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:54:12,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:12,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:12,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:54:12,299 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37011,1676087618428' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-02-11 03:54:12,299 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@443be834[Count = 0] remaining members to acquire global barrier 2023-02-11 03:54:12,299 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-02-11 03:54:12,299 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:12,301 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:12,301 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:12,301 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:12,301 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-02-11 03:54:12,301 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-02-11 03:54:12,301 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,37011,1676087618428' in zk 2023-02-11 03:54:12,301 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:12,302 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 03:54:12,303 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-02-11 03:54:12,303 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:54:12,303 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-02-11 03:54:12,303 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:12,304 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:12,304 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:54:12,304 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:54:12,304 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:54:12,304 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:54:12,305 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:12,305 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:12,305 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:54:12,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:12,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:12,307 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,37011,1676087618428': 2023-02-11 03:54:12,307 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37011,1676087618428' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-02-11 03:54:12,307 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-02-11 03:54:12,307 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 03:54:12,307 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 03:54:12,307 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:12,307 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 03:54:12,309 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:12,309 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:12,309 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:12,309 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:12,309 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:12,309 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:12,309 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:54:12,309 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:54:12,309 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:12,310 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:12,310 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:54:12,310 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:12,310 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:12,311 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:12,311 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:54:12,312 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:12,312 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:12,312 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:12,312 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:54:12,313 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:12,313 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:12,317 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:12,317 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:12,317 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:12,317 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-02-11 03:54:12,317 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 03:54:12,317 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:12,317 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:12,317 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:12,317 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:12,317 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:12,317 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:12,317 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:54:12,318 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-02-11 03:54:12,318 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 03:54:12,318 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:12,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:12,318 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:12,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:12,418 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 03:54:12,419 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 03:54:12,424 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 03:54:12,427 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-02-11 03:54:12,427 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:12,427 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-02-11 03:54:12,427 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:54:12,427 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-02-11 03:54:12,427 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-02-11 03:54:12,427 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:12,427 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:12,428 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:12,429 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:12,429 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:12,429 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:12,429 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:12,429 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 03:54:12,429 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:12,429 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:12,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 03:54:12,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:12,430 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:12,430 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-02-11 03:54:12,430 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:12,430 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-02-11 03:54:12,430 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:54:12,430 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-02-11 03:54:12,430 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 03:54:12,430 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 03:54:12,430 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1676087646502.1e125fab80f689690e5f26dc5627861b. 2023-02-11 03:54:12,430 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1676087646502.1e125fab80f689690e5f26dc5627861b. started... 2023-02-11 03:54:12,431 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 1e125fab80f689690e5f26dc5627861b 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-11 03:54:12,829 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-11 03:54:12,844 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201601030d9f94e53d6f47c5aadb3db9974bb793 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e201601030d9f94e53d6f47c5aadb3db9974bb793 2023-02-11 03:54:13,250 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=33, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMinorCompactionWithWeeklyPolicy/1e125fab80f689690e5f26dc5627861b/.tmp/family1/4276710386524646b2f67cac00c9e122 2023-02-11 03:54:13,255 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMinorCompactionWithWeeklyPolicy/1e125fab80f689690e5f26dc5627861b/.tmp/family1/4276710386524646b2f67cac00c9e122 as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMinorCompactionWithWeeklyPolicy/1e125fab80f689690e5f26dc5627861b/family1/4276710386524646b2f67cac00c9e122 2023-02-11 03:54:13,259 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMinorCompactionWithWeeklyPolicy/1e125fab80f689690e5f26dc5627861b/family1/4276710386524646b2f67cac00c9e122, entries=1, sequenceid=33, filesize=4.9 K 2023-02-11 03:54:13,260 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 1e125fab80f689690e5f26dc5627861b in 829ms, sequenceid=33, compaction requested=true 2023-02-11 03:54:13,260 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 1e125fab80f689690e5f26dc5627861b: 2023-02-11 03:54:13,261 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1676087646502.1e125fab80f689690e5f26dc5627861b. 2023-02-11 03:54:13,261 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 03:54:13,261 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 03:54:13,261 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:13,261 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-02-11 03:54:13,261 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37011,1676087618428' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-02-11 03:54:13,262 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:13,262 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:13,262 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:13,262 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:54:13,262 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:54:13,263 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:13,263 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 03:54:13,263 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:54:13,263 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:54:13,263 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:13,263 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:13,264 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:54:13,264 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37011,1676087618428' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-02-11 03:54:13,264 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3608601f[Count = 0] remaining members to acquire global barrier 2023-02-11 03:54:13,264 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-02-11 03:54:13,264 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:13,265 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:13,265 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:13,265 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:13,265 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-02-11 03:54:13,265 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-02-11 03:54:13,265 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:13,265 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 03:54:13,265 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,37011,1676087618428' in zk 2023-02-11 03:54:13,268 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:13,268 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-02-11 03:54:13,268 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:13,268 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:54:13,268 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:54:13,268 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:54:13,268 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-02-11 03:54:13,268 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:54:13,268 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:54:13,269 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:13,269 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:13,269 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:54:13,269 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:13,269 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:13,270 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,37011,1676087618428': 2023-02-11 03:54:13,270 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37011,1676087618428' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-02-11 03:54:13,270 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-02-11 03:54:13,270 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 03:54:13,270 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 03:54:13,270 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:13,270 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 03:54:13,271 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:13,271 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:13,271 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:13,271 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:54:13,271 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:54:13,271 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:13,271 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:13,272 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:13,272 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:54:13,272 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:13,272 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:13,272 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:13,272 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:13,272 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:13,272 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:54:13,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:13,273 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:13,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:13,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:54:13,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:13,274 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:13,275 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:13,275 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:13,275 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:13,275 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:13,275 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-02-11 03:54:13,275 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 03:54:13,275 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:13,275 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:54:13,275 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:13,275 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:13,275 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:13,276 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-02-11 03:54:13,275 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-02-11 03:54:13,276 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 03:54:13,276 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:13,276 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:13,276 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:13,276 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:13,276 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:13,376 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 03:54:13,377 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 03:54:13,382 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 03:54:13,384 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-02-11 03:54:13,385 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:13,385 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-02-11 03:54:13,385 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:54:13,385 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-02-11 03:54:13,385 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-02-11 03:54:13,386 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:13,386 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:13,387 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:13,387 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:13,387 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:13,388 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:13,388 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:13,388 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 03:54:13,388 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:13,389 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:13,389 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 03:54:13,389 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:13,389 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:13,389 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-02-11 03:54:13,389 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:13,389 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-02-11 03:54:13,389 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:54:13,390 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-02-11 03:54:13,390 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 03:54:13,390 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 03:54:13,390 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1676087646502.1e125fab80f689690e5f26dc5627861b. 2023-02-11 03:54:13,390 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1676087646502.1e125fab80f689690e5f26dc5627861b. started... 2023-02-11 03:54:13,390 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 1e125fab80f689690e5f26dc5627861b 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-11 03:54:13,404 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20150907ecd743f712944bd1913a941c02432d7d to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20150907ecd743f712944bd1913a941c02432d7d 2023-02-11 03:54:13,409 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=37, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMinorCompactionWithWeeklyPolicy/1e125fab80f689690e5f26dc5627861b/.tmp/family1/f8e2de642c8646d3a31ac495a4d3751a 2023-02-11 03:54:13,413 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMinorCompactionWithWeeklyPolicy/1e125fab80f689690e5f26dc5627861b/.tmp/family1/f8e2de642c8646d3a31ac495a4d3751a as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMinorCompactionWithWeeklyPolicy/1e125fab80f689690e5f26dc5627861b/family1/f8e2de642c8646d3a31ac495a4d3751a 2023-02-11 03:54:13,417 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMinorCompactionWithWeeklyPolicy/1e125fab80f689690e5f26dc5627861b/family1/f8e2de642c8646d3a31ac495a4d3751a, entries=1, sequenceid=37, filesize=4.9 K 2023-02-11 03:54:13,418 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 1e125fab80f689690e5f26dc5627861b in 28ms, sequenceid=37, compaction requested=true 2023-02-11 03:54:13,418 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 1e125fab80f689690e5f26dc5627861b: 2023-02-11 03:54:13,418 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1676087646502.1e125fab80f689690e5f26dc5627861b. 2023-02-11 03:54:13,418 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 03:54:13,418 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 03:54:13,418 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:13,418 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-02-11 03:54:13,419 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37011,1676087618428' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-02-11 03:54:13,422 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:13,422 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:13,422 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:13,422 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:54:13,422 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:54:13,422 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:13,422 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 03:54:13,422 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:54:13,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:54:13,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:13,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:13,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:54:13,424 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37011,1676087618428' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-02-11 03:54:13,424 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@355091dd[Count = 0] remaining members to acquire global barrier 2023-02-11 03:54:13,424 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-02-11 03:54:13,424 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:13,427 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:13,427 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:13,428 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:13,428 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-02-11 03:54:13,428 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:13,428 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 03:54:13,428 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-02-11 03:54:13,428 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,37011,1676087618428' in zk 2023-02-11 03:54:13,429 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-02-11 03:54:13,429 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:13,429 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:54:13,429 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:13,429 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-02-11 03:54:13,429 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:54:13,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:54:13,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:54:13,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:54:13,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:13,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:13,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:54:13,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:13,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:13,432 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,37011,1676087618428': 2023-02-11 03:54:13,432 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37011,1676087618428' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-02-11 03:54:13,432 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-02-11 03:54:13,432 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 03:54:13,432 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 03:54:13,432 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:13,432 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 03:54:13,433 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:13,433 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:13,433 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:13,433 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:13,433 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:13,433 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:54:13,433 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:54:13,433 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:13,433 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:13,434 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:13,434 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:54:13,434 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:13,434 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:13,434 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:13,434 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:54:13,434 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:13,435 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:13,435 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:13,435 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:54:13,435 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:13,436 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:13,438 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:13,438 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:13,438 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:13,438 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:13,438 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-02-11 03:54:13,438 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 03:54:13,438 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:13,438 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:13,438 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:13,438 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:13,438 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:13,438 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:13,438 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:13,439 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:13,438 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:54:13,439 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-02-11 03:54:13,439 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 03:54:13,438 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:13,539 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 03:54:13,540 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 03:54:13,545 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 03:54:13,547 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-02-11 03:54:13,547 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:13,547 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-02-11 03:54:13,547 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:54:13,547 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-02-11 03:54:13,547 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-02-11 03:54:13,547 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:13,547 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:13,549 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:13,549 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:13,549 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:13,549 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:13,549 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:13,549 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:13,549 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 03:54:13,549 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:13,550 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 03:54:13,550 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:13,550 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:13,550 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-02-11 03:54:13,550 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:13,550 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-02-11 03:54:13,550 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:54:13,550 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-02-11 03:54:13,551 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 03:54:13,551 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 03:54:13,551 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1676087646502.1e125fab80f689690e5f26dc5627861b. 2023-02-11 03:54:13,551 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1676087646502.1e125fab80f689690e5f26dc5627861b. started... 2023-02-11 03:54:13,551 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 1e125fab80f689690e5f26dc5627861b 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-11 03:54:13,971 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015112075929018f68944f0b19bce40c7e78bed to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e2015112075929018f68944f0b19bce40c7e78bed 2023-02-11 03:54:14,377 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=41, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMinorCompactionWithWeeklyPolicy/1e125fab80f689690e5f26dc5627861b/.tmp/family1/60424da8d09042f4b70ad990a94e1766 2023-02-11 03:54:14,384 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMinorCompactionWithWeeklyPolicy/1e125fab80f689690e5f26dc5627861b/.tmp/family1/60424da8d09042f4b70ad990a94e1766 as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMinorCompactionWithWeeklyPolicy/1e125fab80f689690e5f26dc5627861b/family1/60424da8d09042f4b70ad990a94e1766 2023-02-11 03:54:14,389 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMinorCompactionWithWeeklyPolicy/1e125fab80f689690e5f26dc5627861b/family1/60424da8d09042f4b70ad990a94e1766, entries=1, sequenceid=41, filesize=4.9 K 2023-02-11 03:54:14,390 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 1e125fab80f689690e5f26dc5627861b in 839ms, sequenceid=41, compaction requested=true 2023-02-11 03:54:14,390 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 1e125fab80f689690e5f26dc5627861b: 2023-02-11 03:54:14,390 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1676087646502.1e125fab80f689690e5f26dc5627861b. 2023-02-11 03:54:14,391 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 03:54:14,391 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 03:54:14,391 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:14,391 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-02-11 03:54:14,391 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37011,1676087618428' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-02-11 03:54:14,392 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:14,392 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:14,392 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:14,392 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:54:14,393 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:54:14,393 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:54:14,393 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:14,393 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 03:54:14,394 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:54:14,394 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:14,395 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:14,395 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:54:14,395 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37011,1676087618428' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-02-11 03:54:14,395 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7419dd25[Count = 0] remaining members to acquire global barrier 2023-02-11 03:54:14,395 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-02-11 03:54:14,396 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:14,397 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:14,397 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:14,397 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:14,397 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-02-11 03:54:14,397 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-02-11 03:54:14,397 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,37011,1676087618428' in zk 2023-02-11 03:54:14,397 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:14,397 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 03:54:14,398 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:14,398 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-02-11 03:54:14,399 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:14,399 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:54:14,399 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:54:14,399 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:54:14,399 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-02-11 03:54:14,399 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:54:14,399 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:54:14,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:14,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:14,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:54:14,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:14,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:14,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,37011,1676087618428': 2023-02-11 03:54:14,402 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37011,1676087618428' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-02-11 03:54:14,402 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-02-11 03:54:14,402 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 03:54:14,402 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 03:54:14,402 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:14,402 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 03:54:14,405 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:14,405 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:14,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:14,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:54:14,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:54:14,405 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:14,406 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:14,405 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:14,406 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:14,406 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:14,406 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:54:14,406 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:14,406 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:14,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:14,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:54:14,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:14,408 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:14,408 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:14,408 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:54:14,408 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:14,409 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:14,410 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:14,410 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:14,410 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:14,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:14,410 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-02-11 03:54:14,410 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 03:54:14,410 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:14,410 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:54:14,410 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:14,410 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:14,410 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-02-11 03:54:14,411 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-02-11 03:54:14,411 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:14,411 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:14,411 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:14,411 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:14,411 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 03:54:14,411 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:14,411 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:14,511 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 03:54:14,512 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 03:54:14,513 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=39921] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}] of the table testMinorCompactionWithWeeklyPolicy 2023-02-11 03:54:14,515 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676087621935] procedure2.ProcedureExecutor(1029): Stored pid=26, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE 2023-02-11 03:54:14,516 DEBUG [PEWorker-2] locking.LockProcedure(309): LOCKED pid=26, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE 2023-02-11 03:54:14,516 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676087621935] mob.MobUtils(833): start MOB compaction of files for table='testMinorCompactionWithWeeklyPolicy', column='family1', allFiles=false, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-11 03:54:14,516 INFO [PEWorker-2] procedure2.TimeoutExecutorThread(81): ADDED pid=26, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1676088254516 2023-02-11 03:54:14,517 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676087621935] compactions.PartitionedMobCompactor(142): is allFiles: false 2023-02-11 03:54:14,519 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676087621935] compactions.PartitionedMobCompactor(306): The compaction type is PART_FILES, the request has 0 del files, 9 selected files, and 0 irrelevant files table 'testMinorCompactionWithWeeklyPolicy' and column 'family1' 2023-02-11 03:54:14,519 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676087621935] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMinorCompactionWithWeeklyPolicy' column='family1' 2023-02-11 03:54:14,519 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676087621935] zookeeper.ReadOnlyZKClient(139): Connect 0x641abd90 to 127.0.0.1:53383 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-11 03:54:14,524 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676087621935] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@30904792, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-11 03:54:14,524 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-02-11 03:54:15,372 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160103796ce8e378e34efca63e5b04800ec52f to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20160103796ce8e378e34efca63e5b04800ec52f 2023-02-11 03:54:15,373 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMinorCompactionWithWeeklyPolicy' column='family1' 2023-02-11 03:54:15,386 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-11 03:54:15,387 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:44958, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-11 03:54:15,402 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-02-11 03:54:15,404 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:36360, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-02-11 03:54:15,417 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228/family1/0087cf6982f44626bc1678c3923aa4c1 first=Optional[r3] last=Optional[r6] 2023-02-11 03:54:15,418 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMinorCompactionWithWeeklyPolicy,,1676087646502.1e125fab80f689690e5f26dc5627861b., hostname=jenkins-hbase4.apache.org,37011,1676087618428, seqNum=2 for row with hfile group [{family1,hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228/family1/0087cf6982f44626bc1678c3923aa4c1}] 2023-02-11 03:54:15,423 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.HStore(763): Validating hfile at hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228/family1/0087cf6982f44626bc1678c3923aa4c1 for inclusion in 1e125fab80f689690e5f26dc5627861b/family1 2023-02-11 03:54:15,427 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.HStore(775): HFile bounds: first=r3 last=r6 2023-02-11 03:54:15,427 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.HStore(777): Region bounds: first= last= 2023-02-11 03:54:15,427 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.HRegion(2445): Flush status journal for 1e125fab80f689690e5f26dc5627861b: 2023-02-11 03:54:15,430 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228/family1/0087cf6982f44626bc1678c3923aa4c1 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/staging/jenkins__testMinorCompactionWithWeeklyPolicy__mhs9a8l6257mfe5b650i8kso1mlfbbku9m7rejchldmbu1oodpfi8j77720vstk7/family1/0087cf6982f44626bc1678c3923aa4c1 2023-02-11 03:54:15,433 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/staging/jenkins__testMinorCompactionWithWeeklyPolicy__mhs9a8l6257mfe5b650i8kso1mlfbbku9m7rejchldmbu1oodpfi8j77720vstk7/family1/0087cf6982f44626bc1678c3923aa4c1 as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMinorCompactionWithWeeklyPolicy/1e125fab80f689690e5f26dc5627861b/family1/bb69e6c7788f46a59a020c30f276c743_SeqId_44_ 2023-02-11 03:54:15,433 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.HStore(849): Loaded HFile hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/staging/jenkins__testMinorCompactionWithWeeklyPolicy__mhs9a8l6257mfe5b650i8kso1mlfbbku9m7rejchldmbu1oodpfi8j77720vstk7/family1/0087cf6982f44626bc1678c3923aa4c1 into 1e125fab80f689690e5f26dc5627861b/family1 as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMinorCompactionWithWeeklyPolicy/1e125fab80f689690e5f26dc5627861b/family1/bb69e6c7788f46a59a020c30f276c743_SeqId_44_ - updating store file list. 2023-02-11 03:54:15,438 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.HStore(882): Loaded HFile hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMinorCompactionWithWeeklyPolicy/1e125fab80f689690e5f26dc5627861b/family1/bb69e6c7788f46a59a020c30f276c743_SeqId_44_ into 1e125fab80f689690e5f26dc5627861b/family1 2023-02-11 03:54:15,439 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.HStore(855): Successfully loaded hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/staging/jenkins__testMinorCompactionWithWeeklyPolicy__mhs9a8l6257mfe5b650i8kso1mlfbbku9m7rejchldmbu1oodpfi8j77720vstk7/family1/0087cf6982f44626bc1678c3923aa4c1 into 1e125fab80f689690e5f26dc5627861b/family1 (new location: hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMinorCompactionWithWeeklyPolicy/1e125fab80f689690e5f26dc5627861b/family1/bb69e6c7788f46a59a020c30f276c743_SeqId_44_) 2023-02-11 03:54:15,439 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/staging/jenkins__testMinorCompactionWithWeeklyPolicy__mhs9a8l6257mfe5b650i8kso1mlfbbku9m7rejchldmbu1oodpfi8j77720vstk7/family1/0087cf6982f44626bc1678c3923aa4c1 2023-02-11 03:54:15,439 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] compactions.SortedCompactionPolicy(75): Selecting compaction from 11 store files, 0 compacting, 11 eligible, 16 blocking 2023-02-11 03:54:15,443 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 10 files of size 51214 starting at candidate #1 after considering 44 permutations with 44 in ratio 2023-02-11 03:54:15,443 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.HStore(1912): 1e125fab80f689690e5f26dc5627861b/family1 is initiating minor compaction 2023-02-11 03:54:15,443 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@1654b1de; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-11 03:54:15,443 INFO [RS:0;jenkins-hbase4:37011-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 1e125fab80f689690e5f26dc5627861b/family1 in testMinorCompactionWithWeeklyPolicy,,1676087646502.1e125fab80f689690e5f26dc5627861b. 2023-02-11 03:54:15,443 INFO [RS:0;jenkins-hbase4:37011-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMinorCompactionWithWeeklyPolicy/1e125fab80f689690e5f26dc5627861b/family1/1b6f4852f6e34577a74e2649ed4aa1d5, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMinorCompactionWithWeeklyPolicy/1e125fab80f689690e5f26dc5627861b/family1/08ff73d09b584551af0e99a28732cf57, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMinorCompactionWithWeeklyPolicy/1e125fab80f689690e5f26dc5627861b/family1/66cd78939a4748e0b1c48b19ebc2b58c, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMinorCompactionWithWeeklyPolicy/1e125fab80f689690e5f26dc5627861b/family1/5808fd18d56e42c1ba627a0422c00010, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMinorCompactionWithWeeklyPolicy/1e125fab80f689690e5f26dc5627861b/family1/108050f1493a4b4cac8260464fc349cc, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMinorCompactionWithWeeklyPolicy/1e125fab80f689690e5f26dc5627861b/family1/cd4a04303ec2472c859d660fd8b89609, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMinorCompactionWithWeeklyPolicy/1e125fab80f689690e5f26dc5627861b/family1/4276710386524646b2f67cac00c9e122, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMinorCompactionWithWeeklyPolicy/1e125fab80f689690e5f26dc5627861b/family1/f8e2de642c8646d3a31ac495a4d3751a, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMinorCompactionWithWeeklyPolicy/1e125fab80f689690e5f26dc5627861b/family1/60424da8d09042f4b70ad990a94e1766, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMinorCompactionWithWeeklyPolicy/1e125fab80f689690e5f26dc5627861b/family1/bb69e6c7788f46a59a020c30f276c743_SeqId_44_] into tmpdir=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMinorCompactionWithWeeklyPolicy/1e125fab80f689690e5f26dc5627861b/.tmp, totalSize=50.0 K 2023-02-11 03:54:15,443 DEBUG [RS:0;jenkins-hbase4:37011-shortCompactions-0] compactions.Compactor(207): Compacting 1b6f4852f6e34577a74e2649ed4aa1d5, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=9 2023-02-11 03:54:15,444 DEBUG [RS:0;jenkins-hbase4:37011-shortCompactions-0] compactions.Compactor(207): Compacting 08ff73d09b584551af0e99a28732cf57, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=13 2023-02-11 03:54:15,444 DEBUG [RS:0;jenkins-hbase4:37011-shortCompactions-0] compactions.Compactor(207): Compacting 66cd78939a4748e0b1c48b19ebc2b58c, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=17 2023-02-11 03:54:15,444 DEBUG [RS:0;jenkins-hbase4:37011-shortCompactions-0] compactions.Compactor(207): Compacting 5808fd18d56e42c1ba627a0422c00010, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=21 2023-02-11 03:54:15,444 DEBUG [RS:0;jenkins-hbase4:37011-shortCompactions-0] compactions.Compactor(207): Compacting 108050f1493a4b4cac8260464fc349cc, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=25 2023-02-11 03:54:15,444 DEBUG [RS:0;jenkins-hbase4:37011-shortCompactions-0] compactions.Compactor(207): Compacting cd4a04303ec2472c859d660fd8b89609, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=29 2023-02-11 03:54:15,444 DEBUG [RS:0;jenkins-hbase4:37011-shortCompactions-0] compactions.Compactor(207): Compacting 4276710386524646b2f67cac00c9e122, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=33 2023-02-11 03:54:15,444 DEBUG [RS:0;jenkins-hbase4:37011-shortCompactions-0] compactions.Compactor(207): Compacting f8e2de642c8646d3a31ac495a4d3751a, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=37 2023-02-11 03:54:15,444 DEBUG [RS:0;jenkins-hbase4:37011-shortCompactions-0] compactions.Compactor(207): Compacting 60424da8d09042f4b70ad990a94e1766, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=41 2023-02-11 03:54:15,444 DEBUG [RS:0;jenkins-hbase4:37011-shortCompactions-0] compactions.Compactor(207): Compacting bb69e6c7788f46a59a020c30f276c743_SeqId_44_, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=45 2023-02-11 03:54:15,445 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMinorCompactionWithWeeklyPolicy' column='family1' 2023-02-11 03:54:15,446 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-11 03:54:15,448 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e2015122863a2b2fb23a7458c90fb0f5790ed42a9 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e2015122863a2b2fb23a7458c90fb0f5790ed42a9 2023-02-11 03:54:15,449 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20151231d286abba7c0b49668947e555bc48bc68 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20151231d286abba7c0b49668947e555bc48bc68 2023-02-11 03:54:15,450 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e201601012ac79880d52a4799a9477fda0da5c2ea to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e201601012ac79880d52a4799a9477fda0da5c2ea 2023-02-11 03:54:15,452 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e201601030d9f94e53d6f47c5aadb3db9974bb793 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e201601030d9f94e53d6f47c5aadb3db9974bb793 2023-02-11 03:54:15,452 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMinorCompactionWithWeeklyPolicy 4 expired mob files are deleted 2023-02-11 03:54:15,452 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228' 2023-02-11 03:54:15,452 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-02-11 03:54:15,453 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-02-11 03:54:15,453 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-02-11 03:54:15,453 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-02-11 03:54:15,453 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-02-11 03:54:15,453 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-02-11 03:54:15,453 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-02-11 03:54:15,453 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-02-11 03:54:15,478 INFO [RS:0;jenkins-hbase4:37011-shortCompactions-0] throttle.PressureAwareThroughputController(145): 1e125fab80f689690e5f26dc5627861b#family1#compaction#34 average throughput is 0.68 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-11 03:54:16,294 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151205d053674ac4b049ee9dabfc5c48217df8 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20151205d053674ac4b049ee9dabfc5c48217df8 2023-02-11 03:54:16,295 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMinorCompactionWithWeeklyPolicy' column='family1' 2023-02-11 03:54:16,296 DEBUG [RS:0;jenkins-hbase4:37011-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMinorCompactionWithWeeklyPolicy/1e125fab80f689690e5f26dc5627861b/.tmp/family1/8b7464aa2d71497dac36f0364c553b39 as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMinorCompactionWithWeeklyPolicy/1e125fab80f689690e5f26dc5627861b/family1/8b7464aa2d71497dac36f0364c553b39 2023-02-11 03:54:16,303 INFO [RS:0;jenkins-hbase4:37011-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 10 file(s) in 1e125fab80f689690e5f26dc5627861b/family1 of 1e125fab80f689690e5f26dc5627861b into 8b7464aa2d71497dac36f0364c553b39(size=6.6 K), total size for store is 171.6 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-11 03:54:16,304 DEBUG [RS:0;jenkins-hbase4:37011-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 1e125fab80f689690e5f26dc5627861b: 2023-02-11 03:54:16,304 INFO [RS:0;jenkins-hbase4:37011-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMinorCompactionWithWeeklyPolicy,,1676087646502.1e125fab80f689690e5f26dc5627861b., storeName=family1, fileCount=10, fileSize=50.0 K (5.0 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 5.5 K), priority=2, time=1676087655443; duration=0sec 2023-02-11 03:54:16,304 DEBUG [RS:0;jenkins-hbase4:37011-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-11 03:54:16,322 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130/family1/fafd04d5ecef4719bab228d0ae34a1f3 first=Optional[r1] last=Optional[r2] 2023-02-11 03:54:16,323 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMinorCompactionWithWeeklyPolicy,,1676087646502.1e125fab80f689690e5f26dc5627861b., hostname=jenkins-hbase4.apache.org,37011,1676087618428, seqNum=2 for row with hfile group [{family1,hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130/family1/fafd04d5ecef4719bab228d0ae34a1f3}] 2023-02-11 03:54:16,325 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.HStore(763): Validating hfile at hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130/family1/fafd04d5ecef4719bab228d0ae34a1f3 for inclusion in 1e125fab80f689690e5f26dc5627861b/family1 2023-02-11 03:54:16,328 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.HStore(775): HFile bounds: first=r1 last=r2 2023-02-11 03:54:16,328 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.HStore(777): Region bounds: first= last= 2023-02-11 03:54:16,328 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.HRegion(2445): Flush status journal for 1e125fab80f689690e5f26dc5627861b: 2023-02-11 03:54:16,330 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130/family1/fafd04d5ecef4719bab228d0ae34a1f3 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/staging/jenkins__testMinorCompactionWithWeeklyPolicy__cujo7gbdq9ihf0k102mpns6004s25uo5a58fdvp6r9p60os2uoaj0jffhhhggs4r/family1/fafd04d5ecef4719bab228d0ae34a1f3 2023-02-11 03:54:16,334 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/staging/jenkins__testMinorCompactionWithWeeklyPolicy__cujo7gbdq9ihf0k102mpns6004s25uo5a58fdvp6r9p60os2uoaj0jffhhhggs4r/family1/fafd04d5ecef4719bab228d0ae34a1f3 as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMinorCompactionWithWeeklyPolicy/1e125fab80f689690e5f26dc5627861b/family1/0284f69e84024e9e83d7bf28f8d5a9ea_SeqId_47_ 2023-02-11 03:54:16,334 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.HStore(849): Loaded HFile hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/staging/jenkins__testMinorCompactionWithWeeklyPolicy__cujo7gbdq9ihf0k102mpns6004s25uo5a58fdvp6r9p60os2uoaj0jffhhhggs4r/family1/fafd04d5ecef4719bab228d0ae34a1f3 into 1e125fab80f689690e5f26dc5627861b/family1 as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMinorCompactionWithWeeklyPolicy/1e125fab80f689690e5f26dc5627861b/family1/0284f69e84024e9e83d7bf28f8d5a9ea_SeqId_47_ - updating store file list. 2023-02-11 03:54:16,339 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.HStore(882): Loaded HFile hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMinorCompactionWithWeeklyPolicy/1e125fab80f689690e5f26dc5627861b/family1/0284f69e84024e9e83d7bf28f8d5a9ea_SeqId_47_ into 1e125fab80f689690e5f26dc5627861b/family1 2023-02-11 03:54:16,340 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.HStore(855): Successfully loaded hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/staging/jenkins__testMinorCompactionWithWeeklyPolicy__cujo7gbdq9ihf0k102mpns6004s25uo5a58fdvp6r9p60os2uoaj0jffhhhggs4r/family1/fafd04d5ecef4719bab228d0ae34a1f3 into 1e125fab80f689690e5f26dc5627861b/family1 (new location: hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMinorCompactionWithWeeklyPolicy/1e125fab80f689690e5f26dc5627861b/family1/0284f69e84024e9e83d7bf28f8d5a9ea_SeqId_47_) 2023-02-11 03:54:16,340 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/staging/jenkins__testMinorCompactionWithWeeklyPolicy__cujo7gbdq9ihf0k102mpns6004s25uo5a58fdvp6r9p60os2uoaj0jffhhhggs4r/family1/fafd04d5ecef4719bab228d0ae34a1f3 2023-02-11 03:54:16,340 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-02-11 03:54:16,341 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 181014 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-02-11 03:54:16,342 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.HStore(1912): 1e125fab80f689690e5f26dc5627861b/family1 is initiating minor compaction (all files) 2023-02-11 03:54:16,342 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@42c4b99; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-11 03:54:16,342 INFO [RS:0;jenkins-hbase4:37011-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 1e125fab80f689690e5f26dc5627861b/family1 in testMinorCompactionWithWeeklyPolicy,,1676087646502.1e125fab80f689690e5f26dc5627861b. 2023-02-11 03:54:16,342 INFO [RS:0;jenkins-hbase4:37011-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMinorCompactionWithWeeklyPolicy/1e125fab80f689690e5f26dc5627861b/family1/fc01e6adad6841f89baac037109ec002, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMinorCompactionWithWeeklyPolicy/1e125fab80f689690e5f26dc5627861b/family1/8b7464aa2d71497dac36f0364c553b39, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMinorCompactionWithWeeklyPolicy/1e125fab80f689690e5f26dc5627861b/family1/0284f69e84024e9e83d7bf28f8d5a9ea_SeqId_47_] into tmpdir=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMinorCompactionWithWeeklyPolicy/1e125fab80f689690e5f26dc5627861b/.tmp, totalSize=176.8 K 2023-02-11 03:54:16,342 DEBUG [RS:0;jenkins-hbase4:37011-shortCompactions-0] compactions.Compactor(207): Compacting fc01e6adad6841f89baac037109ec002, keycount=1000, bloomtype=ROW, size=165.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1448878842014 2023-02-11 03:54:16,343 DEBUG [RS:0;jenkins-hbase4:37011-shortCompactions-0] compactions.Compactor(207): Compacting 8b7464aa2d71497dac36f0364c553b39, keycount=9, bloomtype=ROW, size=6.6 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1441621242014 2023-02-11 03:54:16,343 DEBUG [RS:0;jenkins-hbase4:37011-shortCompactions-0] compactions.Compactor(207): Compacting 0284f69e84024e9e83d7bf28f8d5a9ea_SeqId_47_, keycount=2, bloomtype=ROW, size=5.2 K, encoding=NONE, compression=NONE, seqNum=48, earliestPutTs=1448965242014 2023-02-11 03:54:16,344 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMinorCompactionWithWeeklyPolicy' column='family1' 2023-02-11 03:54:16,344 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-11 03:54:16,346 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20151201afe758114a764bcaa3aa3ae8d9864cd7 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20151201afe758114a764bcaa3aa3ae8d9864cd7 2023-02-11 03:54:16,348 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e201512054878b734d81944dc8f0f0dc689e0906d to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e201512054878b734d81944dc8f0f0dc689e0906d 2023-02-11 03:54:16,348 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMinorCompactionWithWeeklyPolicy 2 expired mob files are deleted 2023-02-11 03:54:16,348 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130' 2023-02-11 03:54:16,348 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 2 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-02-11 03:54:16,348 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676087621935] compactions.PartitionedMobCompactor(354): After compaction, there are 5 mob files. table='testMinorCompactionWithWeeklyPolicy' column='family1' 2023-02-11 03:54:16,348 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676087621935] mob.MobUtils(840): end MOB compaction of files for table='testMinorCompactionWithWeeklyPolicy', column='family1', allFiles=false, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-11 03:54:16,350 DEBUG [PEWorker-3] locking.LockProcedure(242): UNLOCKED pid=26, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE 2023-02-11 03:54:16,352 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=26, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE in 1.8360 sec 2023-02-11 03:54:16,359 INFO [RS:0;jenkins-hbase4:37011-shortCompactions-0] throttle.PressureAwareThroughputController(145): 1e125fab80f689690e5f26dc5627861b#family1#compaction#35 average throughput is 25.77 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-11 03:54:16,397 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMinorCompactionWithWeeklyPolicy/1e125fab80f689690e5f26dc5627861b/family1/1b6f4852f6e34577a74e2649ed4aa1d5, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMinorCompactionWithWeeklyPolicy/1e125fab80f689690e5f26dc5627861b/family1/08ff73d09b584551af0e99a28732cf57, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMinorCompactionWithWeeklyPolicy/1e125fab80f689690e5f26dc5627861b/family1/66cd78939a4748e0b1c48b19ebc2b58c, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMinorCompactionWithWeeklyPolicy/1e125fab80f689690e5f26dc5627861b/family1/5808fd18d56e42c1ba627a0422c00010, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMinorCompactionWithWeeklyPolicy/1e125fab80f689690e5f26dc5627861b/family1/108050f1493a4b4cac8260464fc349cc, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMinorCompactionWithWeeklyPolicy/1e125fab80f689690e5f26dc5627861b/family1/cd4a04303ec2472c859d660fd8b89609, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMinorCompactionWithWeeklyPolicy/1e125fab80f689690e5f26dc5627861b/family1/4276710386524646b2f67cac00c9e122, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMinorCompactionWithWeeklyPolicy/1e125fab80f689690e5f26dc5627861b/family1/f8e2de642c8646d3a31ac495a4d3751a, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMinorCompactionWithWeeklyPolicy/1e125fab80f689690e5f26dc5627861b/family1/60424da8d09042f4b70ad990a94e1766, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMinorCompactionWithWeeklyPolicy/1e125fab80f689690e5f26dc5627861b/family1/bb69e6c7788f46a59a020c30f276c743_SeqId_44_] to archive 2023-02-11 03:54:16,397 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-11 03:54:16,399 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMinorCompactionWithWeeklyPolicy/1e125fab80f689690e5f26dc5627861b/family1/1b6f4852f6e34577a74e2649ed4aa1d5 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testMinorCompactionWithWeeklyPolicy/1e125fab80f689690e5f26dc5627861b/family1/1b6f4852f6e34577a74e2649ed4aa1d5 2023-02-11 03:54:16,401 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMinorCompactionWithWeeklyPolicy/1e125fab80f689690e5f26dc5627861b/family1/08ff73d09b584551af0e99a28732cf57 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testMinorCompactionWithWeeklyPolicy/1e125fab80f689690e5f26dc5627861b/family1/08ff73d09b584551af0e99a28732cf57 2023-02-11 03:54:16,402 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMinorCompactionWithWeeklyPolicy/1e125fab80f689690e5f26dc5627861b/family1/66cd78939a4748e0b1c48b19ebc2b58c to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testMinorCompactionWithWeeklyPolicy/1e125fab80f689690e5f26dc5627861b/family1/66cd78939a4748e0b1c48b19ebc2b58c 2023-02-11 03:54:16,403 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMinorCompactionWithWeeklyPolicy/1e125fab80f689690e5f26dc5627861b/family1/5808fd18d56e42c1ba627a0422c00010 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testMinorCompactionWithWeeklyPolicy/1e125fab80f689690e5f26dc5627861b/family1/5808fd18d56e42c1ba627a0422c00010 2023-02-11 03:54:16,405 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMinorCompactionWithWeeklyPolicy/1e125fab80f689690e5f26dc5627861b/family1/108050f1493a4b4cac8260464fc349cc to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testMinorCompactionWithWeeklyPolicy/1e125fab80f689690e5f26dc5627861b/family1/108050f1493a4b4cac8260464fc349cc 2023-02-11 03:54:16,406 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMinorCompactionWithWeeklyPolicy/1e125fab80f689690e5f26dc5627861b/family1/cd4a04303ec2472c859d660fd8b89609 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testMinorCompactionWithWeeklyPolicy/1e125fab80f689690e5f26dc5627861b/family1/cd4a04303ec2472c859d660fd8b89609 2023-02-11 03:54:16,408 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMinorCompactionWithWeeklyPolicy/1e125fab80f689690e5f26dc5627861b/family1/4276710386524646b2f67cac00c9e122 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testMinorCompactionWithWeeklyPolicy/1e125fab80f689690e5f26dc5627861b/family1/4276710386524646b2f67cac00c9e122 2023-02-11 03:54:16,409 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMinorCompactionWithWeeklyPolicy/1e125fab80f689690e5f26dc5627861b/family1/f8e2de642c8646d3a31ac495a4d3751a to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testMinorCompactionWithWeeklyPolicy/1e125fab80f689690e5f26dc5627861b/family1/f8e2de642c8646d3a31ac495a4d3751a 2023-02-11 03:54:16,411 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMinorCompactionWithWeeklyPolicy/1e125fab80f689690e5f26dc5627861b/family1/60424da8d09042f4b70ad990a94e1766 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testMinorCompactionWithWeeklyPolicy/1e125fab80f689690e5f26dc5627861b/family1/60424da8d09042f4b70ad990a94e1766 2023-02-11 03:54:16,412 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMinorCompactionWithWeeklyPolicy/1e125fab80f689690e5f26dc5627861b/family1/bb69e6c7788f46a59a020c30f276c743_SeqId_44_ to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testMinorCompactionWithWeeklyPolicy/1e125fab80f689690e5f26dc5627861b/family1/bb69e6c7788f46a59a020c30f276c743_SeqId_44_ 2023-02-11 03:54:16,442 INFO [Listener at localhost/46205] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMinorCompactionWithWeeklyPolicy Thread=256 (was 251) - Thread LEAK? -, OpenFileDescriptor=608 (was 594) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=231 (was 264), ProcessCount=168 (was 168), AvailableMemoryMB=3401 (was 3570) 2023-02-11 03:54:16,453 INFO [Listener at localhost/46205] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testScannerOnBulkLoadRefHFiles Thread=256, OpenFileDescriptor=608, MaxFileDescriptor=60000, SystemLoadAverage=231, ProcessCount=168, AvailableMemoryMB=3400 2023-02-11 03:54:16,456 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] master.HMaster$4(2102): Client=jenkins//172.31.14.131 create 'testScannerOnBulkLoadRefHFiles', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} 2023-02-11 03:54:16,457 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] procedure2.ProcedureExecutor(1029): Stored pid=27, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testScannerOnBulkLoadRefHFiles 2023-02-11 03:54:16,459 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=27, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testScannerOnBulkLoadRefHFiles execute state=CREATE_TABLE_PRE_OPERATION 2023-02-11 03:54:16,459 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] master.MasterRpcServices(697): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testScannerOnBulkLoadRefHFiles" procId is: 27 2023-02-11 03:54:16,460 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=27, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testScannerOnBulkLoadRefHFiles execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-11 03:54:16,460 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] master.MasterRpcServices(1227): Checking to see if procedure is done pid=27 2023-02-11 03:54:16,462 DEBUG [HFileArchiver-5] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/.tmp/data/default/testScannerOnBulkLoadRefHFiles/1e1751a139bc311156cd44006ffe3506 2023-02-11 03:54:16,462 DEBUG [HFileArchiver-6] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/.tmp/data/default/testScannerOnBulkLoadRefHFiles/2ed5c51addf313895c503de93f35da1a 2023-02-11 03:54:16,462 DEBUG [HFileArchiver-7] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/.tmp/data/default/testScannerOnBulkLoadRefHFiles/af8a134fc8a9b827f534b0bd6e36ea9a 2023-02-11 03:54:16,462 DEBUG [HFileArchiver-5] backup.HFileArchiver(153): Directory hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/.tmp/data/default/testScannerOnBulkLoadRefHFiles/1e1751a139bc311156cd44006ffe3506 empty. 2023-02-11 03:54:16,462 DEBUG [HFileArchiver-7] backup.HFileArchiver(153): Directory hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/.tmp/data/default/testScannerOnBulkLoadRefHFiles/af8a134fc8a9b827f534b0bd6e36ea9a empty. 2023-02-11 03:54:16,462 DEBUG [HFileArchiver-6] backup.HFileArchiver(153): Directory hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/.tmp/data/default/testScannerOnBulkLoadRefHFiles/2ed5c51addf313895c503de93f35da1a empty. 2023-02-11 03:54:16,463 DEBUG [HFileArchiver-5] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/.tmp/data/default/testScannerOnBulkLoadRefHFiles/1e1751a139bc311156cd44006ffe3506 2023-02-11 03:54:16,463 DEBUG [HFileArchiver-7] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/.tmp/data/default/testScannerOnBulkLoadRefHFiles/af8a134fc8a9b827f534b0bd6e36ea9a 2023-02-11 03:54:16,463 DEBUG [HFileArchiver-6] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/.tmp/data/default/testScannerOnBulkLoadRefHFiles/2ed5c51addf313895c503de93f35da1a 2023-02-11 03:54:16,463 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testScannerOnBulkLoadRefHFiles regions 2023-02-11 03:54:16,561 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] master.MasterRpcServices(1227): Checking to see if procedure is done pid=27 2023-02-11 03:54:16,762 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] master.MasterRpcServices(1227): Checking to see if procedure is done pid=27 2023-02-11 03:54:16,872 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/.tmp/data/default/testScannerOnBulkLoadRefHFiles/.tabledesc/.tableinfo.0000000001 2023-02-11 03:54:16,873 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(7671): creating {ENCODED => 1e1751a139bc311156cd44006ffe3506, NAME => 'testScannerOnBulkLoadRefHFiles,,1676087656456.1e1751a139bc311156cd44006ffe3506.', STARTKEY => '', ENDKEY => '1'}, tableDescriptor='testScannerOnBulkLoadRefHFiles', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/.tmp 2023-02-11 03:54:16,873 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(7671): creating {ENCODED => af8a134fc8a9b827f534b0bd6e36ea9a, NAME => 'testScannerOnBulkLoadRefHFiles,2,1676087656456.af8a134fc8a9b827f534b0bd6e36ea9a.', STARTKEY => '2', ENDKEY => ''}, tableDescriptor='testScannerOnBulkLoadRefHFiles', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/.tmp 2023-02-11 03:54:16,874 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(7671): creating {ENCODED => 2ed5c51addf313895c503de93f35da1a, NAME => 'testScannerOnBulkLoadRefHFiles,1,1676087656456.2ed5c51addf313895c503de93f35da1a.', STARTKEY => '1', ENDKEY => '2'}, tableDescriptor='testScannerOnBulkLoadRefHFiles', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/.tmp 2023-02-11 03:54:16,883 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(865): Instantiated testScannerOnBulkLoadRefHFiles,2,1676087656456.af8a134fc8a9b827f534b0bd6e36ea9a.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-11 03:54:16,883 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(1603): Closing af8a134fc8a9b827f534b0bd6e36ea9a, disabling compactions & flushes 2023-02-11 03:54:16,883 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(1625): Closing region testScannerOnBulkLoadRefHFiles,2,1676087656456.af8a134fc8a9b827f534b0bd6e36ea9a. 2023-02-11 03:54:16,883 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(1646): Waiting without time limit for close lock on testScannerOnBulkLoadRefHFiles,2,1676087656456.af8a134fc8a9b827f534b0bd6e36ea9a. 2023-02-11 03:54:16,883 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(1713): Acquired close lock on testScannerOnBulkLoadRefHFiles,2,1676087656456.af8a134fc8a9b827f534b0bd6e36ea9a. after waiting 0 ms 2023-02-11 03:54:16,883 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(1723): Updates disabled for region testScannerOnBulkLoadRefHFiles,2,1676087656456.af8a134fc8a9b827f534b0bd6e36ea9a. 2023-02-11 03:54:16,883 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(1837): Closed testScannerOnBulkLoadRefHFiles,2,1676087656456.af8a134fc8a9b827f534b0bd6e36ea9a. 2023-02-11 03:54:16,883 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(1557): Region close journal for af8a134fc8a9b827f534b0bd6e36ea9a: Waiting for close lock at 1676087656883Disabling compacts and flushes for region at 1676087656883Disabling writes for close at 1676087656883Writing region close event to WAL at 1676087656883Closed at 1676087656883 2023-02-11 03:54:16,886 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(865): Instantiated testScannerOnBulkLoadRefHFiles,,1676087656456.1e1751a139bc311156cd44006ffe3506.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-11 03:54:16,887 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(1603): Closing 1e1751a139bc311156cd44006ffe3506, disabling compactions & flushes 2023-02-11 03:54:16,887 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(1625): Closing region testScannerOnBulkLoadRefHFiles,,1676087656456.1e1751a139bc311156cd44006ffe3506. 2023-02-11 03:54:16,887 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testScannerOnBulkLoadRefHFiles,,1676087656456.1e1751a139bc311156cd44006ffe3506. 2023-02-11 03:54:16,887 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(1713): Acquired close lock on testScannerOnBulkLoadRefHFiles,,1676087656456.1e1751a139bc311156cd44006ffe3506. after waiting 0 ms 2023-02-11 03:54:16,887 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(1723): Updates disabled for region testScannerOnBulkLoadRefHFiles,,1676087656456.1e1751a139bc311156cd44006ffe3506. 2023-02-11 03:54:16,887 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(1837): Closed testScannerOnBulkLoadRefHFiles,,1676087656456.1e1751a139bc311156cd44006ffe3506. 2023-02-11 03:54:16,887 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(1557): Region close journal for 1e1751a139bc311156cd44006ffe3506: Waiting for close lock at 1676087656887Disabling compacts and flushes for region at 1676087656887Disabling writes for close at 1676087656887Writing region close event to WAL at 1676087656887Closed at 1676087656887 2023-02-11 03:54:17,063 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] master.MasterRpcServices(1227): Checking to see if procedure is done pid=27 2023-02-11 03:54:17,183 DEBUG [RS:0;jenkins-hbase4:37011-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMinorCompactionWithWeeklyPolicy/1e125fab80f689690e5f26dc5627861b/.tmp/family1/68839f12429f4c6e80c49894fb88f1d4 as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMinorCompactionWithWeeklyPolicy/1e125fab80f689690e5f26dc5627861b/family1/68839f12429f4c6e80c49894fb88f1d4 2023-02-11 03:54:17,189 INFO [RS:0;jenkins-hbase4:37011-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in 1e125fab80f689690e5f26dc5627861b/family1 of 1e125fab80f689690e5f26dc5627861b into 68839f12429f4c6e80c49894fb88f1d4(size=166.4 K), total size for store is 166.4 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-11 03:54:17,189 DEBUG [RS:0;jenkins-hbase4:37011-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 1e125fab80f689690e5f26dc5627861b: 2023-02-11 03:54:17,189 INFO [RS:0;jenkins-hbase4:37011-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMinorCompactionWithWeeklyPolicy,,1676087646502.1e125fab80f689690e5f26dc5627861b., storeName=family1, fileCount=3, fileSize=176.8 K (165.1 K, 6.6 K, 5.2 K), priority=2, time=1676087656342; duration=0sec 2023-02-11 03:54:17,190 DEBUG [RS:0;jenkins-hbase4:37011-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-11 03:54:17,199 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMinorCompactionWithWeeklyPolicy/1e125fab80f689690e5f26dc5627861b/family1/fc01e6adad6841f89baac037109ec002, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMinorCompactionWithWeeklyPolicy/1e125fab80f689690e5f26dc5627861b/family1/8b7464aa2d71497dac36f0364c553b39, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMinorCompactionWithWeeklyPolicy/1e125fab80f689690e5f26dc5627861b/family1/0284f69e84024e9e83d7bf28f8d5a9ea_SeqId_47_] to archive 2023-02-11 03:54:17,200 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-11 03:54:17,203 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMinorCompactionWithWeeklyPolicy/1e125fab80f689690e5f26dc5627861b/family1/fc01e6adad6841f89baac037109ec002 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testMinorCompactionWithWeeklyPolicy/1e125fab80f689690e5f26dc5627861b/family1/fc01e6adad6841f89baac037109ec002 2023-02-11 03:54:17,205 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMinorCompactionWithWeeklyPolicy/1e125fab80f689690e5f26dc5627861b/family1/8b7464aa2d71497dac36f0364c553b39 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testMinorCompactionWithWeeklyPolicy/1e125fab80f689690e5f26dc5627861b/family1/8b7464aa2d71497dac36f0364c553b39 2023-02-11 03:54:17,206 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMinorCompactionWithWeeklyPolicy/1e125fab80f689690e5f26dc5627861b/family1/0284f69e84024e9e83d7bf28f8d5a9ea_SeqId_47_ to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testMinorCompactionWithWeeklyPolicy/1e125fab80f689690e5f26dc5627861b/family1/0284f69e84024e9e83d7bf28f8d5a9ea_SeqId_47_ 2023-02-11 03:54:17,300 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(865): Instantiated testScannerOnBulkLoadRefHFiles,1,1676087656456.2ed5c51addf313895c503de93f35da1a.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-11 03:54:17,300 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(1603): Closing 2ed5c51addf313895c503de93f35da1a, disabling compactions & flushes 2023-02-11 03:54:17,301 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(1625): Closing region testScannerOnBulkLoadRefHFiles,1,1676087656456.2ed5c51addf313895c503de93f35da1a. 2023-02-11 03:54:17,301 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(1646): Waiting without time limit for close lock on testScannerOnBulkLoadRefHFiles,1,1676087656456.2ed5c51addf313895c503de93f35da1a. 2023-02-11 03:54:17,301 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(1713): Acquired close lock on testScannerOnBulkLoadRefHFiles,1,1676087656456.2ed5c51addf313895c503de93f35da1a. after waiting 0 ms 2023-02-11 03:54:17,301 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(1723): Updates disabled for region testScannerOnBulkLoadRefHFiles,1,1676087656456.2ed5c51addf313895c503de93f35da1a. 2023-02-11 03:54:17,301 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(1837): Closed testScannerOnBulkLoadRefHFiles,1,1676087656456.2ed5c51addf313895c503de93f35da1a. 2023-02-11 03:54:17,301 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(1557): Region close journal for 2ed5c51addf313895c503de93f35da1a: Waiting for close lock at 1676087657300Disabling compacts and flushes for region at 1676087657300Disabling writes for close at 1676087657301 (+1 ms)Writing region close event to WAL at 1676087657301Closed at 1676087657301 2023-02-11 03:54:17,303 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=27, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testScannerOnBulkLoadRefHFiles execute state=CREATE_TABLE_ADD_TO_META 2023-02-11 03:54:17,305 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testScannerOnBulkLoadRefHFiles,2,1676087656456.af8a134fc8a9b827f534b0bd6e36ea9a.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1676087657304"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676087657304"}]},"ts":"1676087657304"} 2023-02-11 03:54:17,305 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testScannerOnBulkLoadRefHFiles,,1676087656456.1e1751a139bc311156cd44006ffe3506.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1676087657304"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676087657304"}]},"ts":"1676087657304"} 2023-02-11 03:54:17,305 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testScannerOnBulkLoadRefHFiles,1,1676087656456.2ed5c51addf313895c503de93f35da1a.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1676087657304"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676087657304"}]},"ts":"1676087657304"} 2023-02-11 03:54:17,309 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 3 regions to meta. 2023-02-11 03:54:17,310 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=27, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testScannerOnBulkLoadRefHFiles execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-11 03:54:17,310 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testScannerOnBulkLoadRefHFiles","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676087657310"}]},"ts":"1676087657310"} 2023-02-11 03:54:17,311 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testScannerOnBulkLoadRefHFiles, state=ENABLING in hbase:meta 2023-02-11 03:54:17,316 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=28, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=1e1751a139bc311156cd44006ffe3506, ASSIGN}, {pid=29, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=2ed5c51addf313895c503de93f35da1a, ASSIGN}, {pid=30, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=af8a134fc8a9b827f534b0bd6e36ea9a, ASSIGN}] 2023-02-11 03:54:17,320 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=1e1751a139bc311156cd44006ffe3506, ASSIGN 2023-02-11 03:54:17,320 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=29, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=2ed5c51addf313895c503de93f35da1a, ASSIGN 2023-02-11 03:54:17,321 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=30, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=af8a134fc8a9b827f534b0bd6e36ea9a, ASSIGN 2023-02-11 03:54:17,321 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=28, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=1e1751a139bc311156cd44006ffe3506, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,37011,1676087618428; forceNewPlan=false, retain=false 2023-02-11 03:54:17,321 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=29, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=2ed5c51addf313895c503de93f35da1a, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,37011,1676087618428; forceNewPlan=false, retain=false 2023-02-11 03:54:17,322 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=30, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=af8a134fc8a9b827f534b0bd6e36ea9a, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,37011,1676087618428; forceNewPlan=false, retain=false 2023-02-11 03:54:17,473 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=30 updating hbase:meta row=af8a134fc8a9b827f534b0bd6e36ea9a, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:17,473 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=29 updating hbase:meta row=2ed5c51addf313895c503de93f35da1a, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:17,473 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=28 updating hbase:meta row=1e1751a139bc311156cd44006ffe3506, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:17,474 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testScannerOnBulkLoadRefHFiles,2,1676087656456.af8a134fc8a9b827f534b0bd6e36ea9a.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1676087657473"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676087657473"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676087657473"}]},"ts":"1676087657473"} 2023-02-11 03:54:17,474 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testScannerOnBulkLoadRefHFiles,1,1676087656456.2ed5c51addf313895c503de93f35da1a.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1676087657473"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676087657473"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676087657473"}]},"ts":"1676087657473"} 2023-02-11 03:54:17,474 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testScannerOnBulkLoadRefHFiles,,1676087656456.1e1751a139bc311156cd44006ffe3506.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1676087657473"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676087657473"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676087657473"}]},"ts":"1676087657473"} 2023-02-11 03:54:17,475 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=31, ppid=30, state=RUNNABLE; OpenRegionProcedure af8a134fc8a9b827f534b0bd6e36ea9a, server=jenkins-hbase4.apache.org,37011,1676087618428}] 2023-02-11 03:54:17,476 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=32, ppid=29, state=RUNNABLE; OpenRegionProcedure 2ed5c51addf313895c503de93f35da1a, server=jenkins-hbase4.apache.org,37011,1676087618428}] 2023-02-11 03:54:17,477 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=33, ppid=28, state=RUNNABLE; OpenRegionProcedure 1e1751a139bc311156cd44006ffe3506, server=jenkins-hbase4.apache.org,37011,1676087618428}] 2023-02-11 03:54:17,565 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] master.MasterRpcServices(1227): Checking to see if procedure is done pid=27 2023-02-11 03:54:17,631 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testScannerOnBulkLoadRefHFiles,1,1676087656456.2ed5c51addf313895c503de93f35da1a. 2023-02-11 03:54:17,631 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 2ed5c51addf313895c503de93f35da1a, NAME => 'testScannerOnBulkLoadRefHFiles,1,1676087656456.2ed5c51addf313895c503de93f35da1a.', STARTKEY => '1', ENDKEY => '2'} 2023-02-11 03:54:17,631 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testScannerOnBulkLoadRefHFiles 2ed5c51addf313895c503de93f35da1a 2023-02-11 03:54:17,631 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testScannerOnBulkLoadRefHFiles,1,1676087656456.2ed5c51addf313895c503de93f35da1a.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-11 03:54:17,631 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 2ed5c51addf313895c503de93f35da1a 2023-02-11 03:54:17,631 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 2ed5c51addf313895c503de93f35da1a 2023-02-11 03:54:17,633 INFO [StoreOpener-2ed5c51addf313895c503de93f35da1a-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 2ed5c51addf313895c503de93f35da1a 2023-02-11 03:54:17,634 DEBUG [StoreOpener-2ed5c51addf313895c503de93f35da1a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testScannerOnBulkLoadRefHFiles/2ed5c51addf313895c503de93f35da1a/family1 2023-02-11 03:54:17,634 DEBUG [StoreOpener-2ed5c51addf313895c503de93f35da1a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testScannerOnBulkLoadRefHFiles/2ed5c51addf313895c503de93f35da1a/family1 2023-02-11 03:54:17,635 INFO [StoreOpener-2ed5c51addf313895c503de93f35da1a-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 2ed5c51addf313895c503de93f35da1a columnFamilyName family1 2023-02-11 03:54:17,636 INFO [StoreOpener-2ed5c51addf313895c503de93f35da1a-1] regionserver.HStore(310): Store=2ed5c51addf313895c503de93f35da1a/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 03:54:17,637 INFO [StoreOpener-2ed5c51addf313895c503de93f35da1a-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 2ed5c51addf313895c503de93f35da1a 2023-02-11 03:54:17,638 DEBUG [StoreOpener-2ed5c51addf313895c503de93f35da1a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testScannerOnBulkLoadRefHFiles/2ed5c51addf313895c503de93f35da1a/family2 2023-02-11 03:54:17,638 DEBUG [StoreOpener-2ed5c51addf313895c503de93f35da1a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testScannerOnBulkLoadRefHFiles/2ed5c51addf313895c503de93f35da1a/family2 2023-02-11 03:54:17,638 INFO [StoreOpener-2ed5c51addf313895c503de93f35da1a-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 2ed5c51addf313895c503de93f35da1a columnFamilyName family2 2023-02-11 03:54:17,640 INFO [StoreOpener-2ed5c51addf313895c503de93f35da1a-1] regionserver.HStore(310): Store=2ed5c51addf313895c503de93f35da1a/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 03:54:17,641 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testScannerOnBulkLoadRefHFiles/2ed5c51addf313895c503de93f35da1a 2023-02-11 03:54:17,641 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testScannerOnBulkLoadRefHFiles/2ed5c51addf313895c503de93f35da1a 2023-02-11 03:54:17,643 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testScannerOnBulkLoadRefHFiles descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-11 03:54:17,644 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 2ed5c51addf313895c503de93f35da1a 2023-02-11 03:54:17,646 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testScannerOnBulkLoadRefHFiles/2ed5c51addf313895c503de93f35da1a/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-11 03:54:17,646 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 2ed5c51addf313895c503de93f35da1a; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=75152186, jitterRate=0.11985483765602112}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-11 03:54:17,647 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 2ed5c51addf313895c503de93f35da1a: Running coprocessor pre-open hook at 1676087657632Writing region info on filesystem at 1676087657632Initializing all the Stores at 1676087657632Instantiating store for column family {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1676087657632Instantiating store for column family {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1676087657632Cleaning up temporary data from old regions at 1676087657641 (+9 ms)Cleaning up detritus from prior splits at 1676087657642 (+1 ms)Running coprocessor post-open hooks at 1676087657646 (+4 ms)Region opened successfully at 1676087657647 (+1 ms) 2023-02-11 03:54:17,647 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testScannerOnBulkLoadRefHFiles,1,1676087656456.2ed5c51addf313895c503de93f35da1a., pid=32, masterSystemTime=1676087657628 2023-02-11 03:54:17,649 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testScannerOnBulkLoadRefHFiles,1,1676087656456.2ed5c51addf313895c503de93f35da1a. 2023-02-11 03:54:17,649 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testScannerOnBulkLoadRefHFiles,1,1676087656456.2ed5c51addf313895c503de93f35da1a. 2023-02-11 03:54:17,649 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testScannerOnBulkLoadRefHFiles,,1676087656456.1e1751a139bc311156cd44006ffe3506. 2023-02-11 03:54:17,649 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 1e1751a139bc311156cd44006ffe3506, NAME => 'testScannerOnBulkLoadRefHFiles,,1676087656456.1e1751a139bc311156cd44006ffe3506.', STARTKEY => '', ENDKEY => '1'} 2023-02-11 03:54:17,650 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=29 updating hbase:meta row=2ed5c51addf313895c503de93f35da1a, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:17,650 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testScannerOnBulkLoadRefHFiles 1e1751a139bc311156cd44006ffe3506 2023-02-11 03:54:17,650 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testScannerOnBulkLoadRefHFiles,1,1676087656456.2ed5c51addf313895c503de93f35da1a.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1676087657650"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676087657650"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676087657650"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676087657650"}]},"ts":"1676087657650"} 2023-02-11 03:54:17,650 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testScannerOnBulkLoadRefHFiles,,1676087656456.1e1751a139bc311156cd44006ffe3506.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-11 03:54:17,650 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 1e1751a139bc311156cd44006ffe3506 2023-02-11 03:54:17,650 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 1e1751a139bc311156cd44006ffe3506 2023-02-11 03:54:17,651 INFO [StoreOpener-1e1751a139bc311156cd44006ffe3506-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 1e1751a139bc311156cd44006ffe3506 2023-02-11 03:54:17,653 DEBUG [StoreOpener-1e1751a139bc311156cd44006ffe3506-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testScannerOnBulkLoadRefHFiles/1e1751a139bc311156cd44006ffe3506/family1 2023-02-11 03:54:17,653 DEBUG [StoreOpener-1e1751a139bc311156cd44006ffe3506-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testScannerOnBulkLoadRefHFiles/1e1751a139bc311156cd44006ffe3506/family1 2023-02-11 03:54:17,653 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=32, resume processing ppid=29 2023-02-11 03:54:17,654 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=32, ppid=29, state=SUCCESS; OpenRegionProcedure 2ed5c51addf313895c503de93f35da1a, server=jenkins-hbase4.apache.org,37011,1676087618428 in 175 msec 2023-02-11 03:54:17,654 INFO [StoreOpener-1e1751a139bc311156cd44006ffe3506-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1e1751a139bc311156cd44006ffe3506 columnFamilyName family1 2023-02-11 03:54:17,655 INFO [StoreOpener-1e1751a139bc311156cd44006ffe3506-1] regionserver.HStore(310): Store=1e1751a139bc311156cd44006ffe3506/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 03:54:17,655 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=29, ppid=27, state=SUCCESS; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=2ed5c51addf313895c503de93f35da1a, ASSIGN in 338 msec 2023-02-11 03:54:17,655 INFO [StoreOpener-1e1751a139bc311156cd44006ffe3506-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 1e1751a139bc311156cd44006ffe3506 2023-02-11 03:54:17,657 DEBUG [StoreOpener-1e1751a139bc311156cd44006ffe3506-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testScannerOnBulkLoadRefHFiles/1e1751a139bc311156cd44006ffe3506/family2 2023-02-11 03:54:17,657 DEBUG [StoreOpener-1e1751a139bc311156cd44006ffe3506-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testScannerOnBulkLoadRefHFiles/1e1751a139bc311156cd44006ffe3506/family2 2023-02-11 03:54:17,658 INFO [StoreOpener-1e1751a139bc311156cd44006ffe3506-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1e1751a139bc311156cd44006ffe3506 columnFamilyName family2 2023-02-11 03:54:17,659 INFO [StoreOpener-1e1751a139bc311156cd44006ffe3506-1] regionserver.HStore(310): Store=1e1751a139bc311156cd44006ffe3506/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 03:54:17,660 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testScannerOnBulkLoadRefHFiles/1e1751a139bc311156cd44006ffe3506 2023-02-11 03:54:17,660 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testScannerOnBulkLoadRefHFiles/1e1751a139bc311156cd44006ffe3506 2023-02-11 03:54:17,661 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testScannerOnBulkLoadRefHFiles descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-11 03:54:17,662 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 1e1751a139bc311156cd44006ffe3506 2023-02-11 03:54:17,664 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testScannerOnBulkLoadRefHFiles/1e1751a139bc311156cd44006ffe3506/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-11 03:54:17,664 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 1e1751a139bc311156cd44006ffe3506; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=64723426, jitterRate=-0.035545796155929565}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-11 03:54:17,664 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 1e1751a139bc311156cd44006ffe3506: Running coprocessor pre-open hook at 1676087657650Writing region info on filesystem at 1676087657650Initializing all the Stores at 1676087657651 (+1 ms)Instantiating store for column family {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1676087657651Instantiating store for column family {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1676087657651Cleaning up temporary data from old regions at 1676087657661 (+10 ms)Cleaning up detritus from prior splits at 1676087657661Running coprocessor post-open hooks at 1676087657664 (+3 ms)Region opened successfully at 1676087657664 2023-02-11 03:54:17,665 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testScannerOnBulkLoadRefHFiles,,1676087656456.1e1751a139bc311156cd44006ffe3506., pid=33, masterSystemTime=1676087657628 2023-02-11 03:54:17,666 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testScannerOnBulkLoadRefHFiles,,1676087656456.1e1751a139bc311156cd44006ffe3506. 2023-02-11 03:54:17,667 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testScannerOnBulkLoadRefHFiles,,1676087656456.1e1751a139bc311156cd44006ffe3506. 2023-02-11 03:54:17,667 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testScannerOnBulkLoadRefHFiles,2,1676087656456.af8a134fc8a9b827f534b0bd6e36ea9a. 2023-02-11 03:54:17,667 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => af8a134fc8a9b827f534b0bd6e36ea9a, NAME => 'testScannerOnBulkLoadRefHFiles,2,1676087656456.af8a134fc8a9b827f534b0bd6e36ea9a.', STARTKEY => '2', ENDKEY => ''} 2023-02-11 03:54:17,667 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testScannerOnBulkLoadRefHFiles af8a134fc8a9b827f534b0bd6e36ea9a 2023-02-11 03:54:17,667 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=28 updating hbase:meta row=1e1751a139bc311156cd44006ffe3506, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:17,667 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testScannerOnBulkLoadRefHFiles,2,1676087656456.af8a134fc8a9b827f534b0bd6e36ea9a.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-11 03:54:17,667 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for af8a134fc8a9b827f534b0bd6e36ea9a 2023-02-11 03:54:17,667 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for af8a134fc8a9b827f534b0bd6e36ea9a 2023-02-11 03:54:17,667 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testScannerOnBulkLoadRefHFiles,,1676087656456.1e1751a139bc311156cd44006ffe3506.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1676087657667"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676087657667"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676087657667"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676087657667"}]},"ts":"1676087657667"} 2023-02-11 03:54:17,668 INFO [StoreOpener-af8a134fc8a9b827f534b0bd6e36ea9a-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region af8a134fc8a9b827f534b0bd6e36ea9a 2023-02-11 03:54:17,670 DEBUG [StoreOpener-af8a134fc8a9b827f534b0bd6e36ea9a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testScannerOnBulkLoadRefHFiles/af8a134fc8a9b827f534b0bd6e36ea9a/family1 2023-02-11 03:54:17,670 DEBUG [StoreOpener-af8a134fc8a9b827f534b0bd6e36ea9a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testScannerOnBulkLoadRefHFiles/af8a134fc8a9b827f534b0bd6e36ea9a/family1 2023-02-11 03:54:17,670 INFO [StoreOpener-af8a134fc8a9b827f534b0bd6e36ea9a-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region af8a134fc8a9b827f534b0bd6e36ea9a columnFamilyName family1 2023-02-11 03:54:17,670 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=33, resume processing ppid=28 2023-02-11 03:54:17,670 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=33, ppid=28, state=SUCCESS; OpenRegionProcedure 1e1751a139bc311156cd44006ffe3506, server=jenkins-hbase4.apache.org,37011,1676087618428 in 192 msec 2023-02-11 03:54:17,671 INFO [StoreOpener-af8a134fc8a9b827f534b0bd6e36ea9a-1] regionserver.HStore(310): Store=af8a134fc8a9b827f534b0bd6e36ea9a/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 03:54:17,671 INFO [StoreOpener-af8a134fc8a9b827f534b0bd6e36ea9a-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region af8a134fc8a9b827f534b0bd6e36ea9a 2023-02-11 03:54:17,671 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=28, ppid=27, state=SUCCESS; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=1e1751a139bc311156cd44006ffe3506, ASSIGN in 354 msec 2023-02-11 03:54:17,672 DEBUG [StoreOpener-af8a134fc8a9b827f534b0bd6e36ea9a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testScannerOnBulkLoadRefHFiles/af8a134fc8a9b827f534b0bd6e36ea9a/family2 2023-02-11 03:54:17,672 DEBUG [StoreOpener-af8a134fc8a9b827f534b0bd6e36ea9a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testScannerOnBulkLoadRefHFiles/af8a134fc8a9b827f534b0bd6e36ea9a/family2 2023-02-11 03:54:17,673 INFO [StoreOpener-af8a134fc8a9b827f534b0bd6e36ea9a-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region af8a134fc8a9b827f534b0bd6e36ea9a columnFamilyName family2 2023-02-11 03:54:17,673 INFO [StoreOpener-af8a134fc8a9b827f534b0bd6e36ea9a-1] regionserver.HStore(310): Store=af8a134fc8a9b827f534b0bd6e36ea9a/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 03:54:17,674 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testScannerOnBulkLoadRefHFiles/af8a134fc8a9b827f534b0bd6e36ea9a 2023-02-11 03:54:17,674 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testScannerOnBulkLoadRefHFiles/af8a134fc8a9b827f534b0bd6e36ea9a 2023-02-11 03:54:17,676 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testScannerOnBulkLoadRefHFiles descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-11 03:54:17,677 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for af8a134fc8a9b827f534b0bd6e36ea9a 2023-02-11 03:54:17,678 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testScannerOnBulkLoadRefHFiles/af8a134fc8a9b827f534b0bd6e36ea9a/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-11 03:54:17,678 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened af8a134fc8a9b827f534b0bd6e36ea9a; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=72562905, jitterRate=0.08127154409885406}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-11 03:54:17,678 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for af8a134fc8a9b827f534b0bd6e36ea9a: Running coprocessor pre-open hook at 1676087657667Writing region info on filesystem at 1676087657667Initializing all the Stores at 1676087657668 (+1 ms)Instantiating store for column family {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1676087657668Instantiating store for column family {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1676087657668Cleaning up temporary data from old regions at 1676087657675 (+7 ms)Cleaning up detritus from prior splits at 1676087657675Running coprocessor post-open hooks at 1676087657678 (+3 ms)Region opened successfully at 1676087657678 2023-02-11 03:54:17,679 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testScannerOnBulkLoadRefHFiles,2,1676087656456.af8a134fc8a9b827f534b0bd6e36ea9a., pid=31, masterSystemTime=1676087657628 2023-02-11 03:54:17,680 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testScannerOnBulkLoadRefHFiles,2,1676087656456.af8a134fc8a9b827f534b0bd6e36ea9a. 2023-02-11 03:54:17,680 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testScannerOnBulkLoadRefHFiles,2,1676087656456.af8a134fc8a9b827f534b0bd6e36ea9a. 2023-02-11 03:54:17,681 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=30 updating hbase:meta row=af8a134fc8a9b827f534b0bd6e36ea9a, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:17,681 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testScannerOnBulkLoadRefHFiles,2,1676087656456.af8a134fc8a9b827f534b0bd6e36ea9a.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1676087657681"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676087657681"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676087657681"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676087657681"}]},"ts":"1676087657681"} 2023-02-11 03:54:17,684 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=31, resume processing ppid=30 2023-02-11 03:54:17,684 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=31, ppid=30, state=SUCCESS; OpenRegionProcedure af8a134fc8a9b827f534b0bd6e36ea9a, server=jenkins-hbase4.apache.org,37011,1676087618428 in 207 msec 2023-02-11 03:54:17,685 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=30, resume processing ppid=27 2023-02-11 03:54:17,686 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=30, ppid=27, state=SUCCESS; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=af8a134fc8a9b827f534b0bd6e36ea9a, ASSIGN in 368 msec 2023-02-11 03:54:17,686 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=27, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testScannerOnBulkLoadRefHFiles execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-11 03:54:17,686 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testScannerOnBulkLoadRefHFiles","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676087657686"}]},"ts":"1676087657686"} 2023-02-11 03:54:17,687 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=testScannerOnBulkLoadRefHFiles, state=ENABLED in hbase:meta 2023-02-11 03:54:17,689 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=27, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testScannerOnBulkLoadRefHFiles execute state=CREATE_TABLE_POST_OPERATION 2023-02-11 03:54:17,691 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=27, state=SUCCESS; CreateTableProcedure table=testScannerOnBulkLoadRefHFiles in 1.2330 sec 2023-02-11 03:54:18,566 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] master.MasterRpcServices(1227): Checking to see if procedure is done pid=27 2023-02-11 03:54:18,566 INFO [Listener at localhost/46205] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testScannerOnBulkLoadRefHFiles, procId: 27 completed 2023-02-11 03:54:18,573 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 03:54:18,574 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] procedure.ProcedureCoordinator(165): Submitting procedure testScannerOnBulkLoadRefHFiles 2023-02-11 03:54:18,574 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testScannerOnBulkLoadRefHFiles' 2023-02-11 03:54:18,574 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:54:18,575 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testScannerOnBulkLoadRefHFiles' starting 'acquire' 2023-02-11 03:54:18,575 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testScannerOnBulkLoadRefHFiles', kicking off acquire phase on members. 2023-02-11 03:54:18,575 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-02-11 03:54:18,575 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-02-11 03:54:18,576 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:18,576 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:18,576 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:18,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:18,577 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:18,577 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 03:54:18,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-02-11 03:54:18,577 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-02-11 03:54:18,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 03:54:18,578 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-02-11 03:54:18,578 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testScannerOnBulkLoadRefHFiles 2023-02-11 03:54:18,578 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testScannerOnBulkLoadRefHFiles 2023-02-11 03:54:18,578 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testScannerOnBulkLoadRefHFiles' with timeout 60000ms 2023-02-11 03:54:18,578 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:54:18,578 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testScannerOnBulkLoadRefHFiles' starting 'acquire' stage 2023-02-11 03:54:18,578 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testScannerOnBulkLoadRefHFiles,,1676087656456.1e1751a139bc311156cd44006ffe3506. 2023-02-11 03:54:18,578 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testScannerOnBulkLoadRefHFiles,2,1676087656456.af8a134fc8a9b827f534b0bd6e36ea9a. 2023-02-11 03:54:18,578 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testScannerOnBulkLoadRefHFiles,1,1676087656456.2ed5c51addf313895c503de93f35da1a. 2023-02-11 03:54:18,578 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testScannerOnBulkLoadRefHFiles,1,1676087656456.2ed5c51addf313895c503de93f35da1a. started... 2023-02-11 03:54:18,578 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testScannerOnBulkLoadRefHFiles,,1676087656456.1e1751a139bc311156cd44006ffe3506. started... 2023-02-11 03:54:18,579 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-2] regionserver.HRegion(2445): Flush status journal for 2ed5c51addf313895c503de93f35da1a: 2023-02-11 03:54:18,578 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-02-11 03:54:18,580 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 1e1751a139bc311156cd44006ffe3506: 2023-02-11 03:54:18,580 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 03:54:18,580 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testScannerOnBulkLoadRefHFiles,,1676087656456.1e1751a139bc311156cd44006ffe3506. 2023-02-11 03:54:18,579 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testScannerOnBulkLoadRefHFiles,1,1676087656456.2ed5c51addf313895c503de93f35da1a. 2023-02-11 03:54:18,580 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-02-11 03:54:18,578 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testScannerOnBulkLoadRefHFiles,2,1676087656456.af8a134fc8a9b827f534b0bd6e36ea9a. started... 2023-02-11 03:54:18,580 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-02-11 03:54:18,581 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-1] regionserver.HRegion(2744): Flushing af8a134fc8a9b827f534b0bd6e36ea9a 2/2 column families, dataSize=48 B heapSize=632 B 2023-02-11 03:54:18,996 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230211db356e8ae10b496fbd00b289cbd1c3e0 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testScannerOnBulkLoadRefHFiles/9a980ee3ec24e1d8bfa59c2e531fd315/family1/c81e728d9d4c2f636f067f89cc14862c20230211db356e8ae10b496fbd00b289cbd1c3e0 2023-02-11 03:54:19,403 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=48, hasBloomFilter=true, into tmp file hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testScannerOnBulkLoadRefHFiles/af8a134fc8a9b827f534b0bd6e36ea9a/.tmp/family1/a771c94e53fe4dc3adb4c25c4f94cad0 2023-02-11 03:54:19,408 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testScannerOnBulkLoadRefHFiles/af8a134fc8a9b827f534b0bd6e36ea9a/.tmp/family1/a771c94e53fe4dc3adb4c25c4f94cad0 as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testScannerOnBulkLoadRefHFiles/af8a134fc8a9b827f534b0bd6e36ea9a/family1/a771c94e53fe4dc3adb4c25c4f94cad0 2023-02-11 03:54:19,412 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testScannerOnBulkLoadRefHFiles/af8a134fc8a9b827f534b0bd6e36ea9a/family1/a771c94e53fe4dc3adb4c25c4f94cad0, entries=1, sequenceid=5, filesize=4.9 K 2023-02-11 03:54:19,413 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-1] regionserver.HRegion(2947): Finished flush of dataSize ~48 B/48, heapSize ~360 B/360, currentSize=0 B/0 for af8a134fc8a9b827f534b0bd6e36ea9a in 832ms, sequenceid=5, compaction requested=false 2023-02-11 03:54:19,413 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testScannerOnBulkLoadRefHFiles' 2023-02-11 03:54:19,414 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-1] regionserver.HRegion(2445): Flush status journal for af8a134fc8a9b827f534b0bd6e36ea9a: 2023-02-11 03:54:19,414 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testScannerOnBulkLoadRefHFiles,2,1676087656456.af8a134fc8a9b827f534b0bd6e36ea9a. 2023-02-11 03:54:19,414 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-02-11 03:54:19,414 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-02-11 03:54:19,414 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:19,414 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testScannerOnBulkLoadRefHFiles' locally acquired 2023-02-11 03:54:19,414 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37011,1676087618428' joining acquired barrier for procedure (testScannerOnBulkLoadRefHFiles) in zk 2023-02-11 03:54:19,415 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:19,415 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-02-11 03:54:19,415 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:19,415 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:54:19,415 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:54:19,416 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-02-11 03:54:19,416 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testScannerOnBulkLoadRefHFiles' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 03:54:19,416 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:54:19,416 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:54:19,416 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-02-11 03:54:19,416 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:19,417 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:54:19,417 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37011,1676087618428' joining acquired barrier for procedure 'testScannerOnBulkLoadRefHFiles' on coordinator 2023-02-11 03:54:19,417 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6f78f1da[Count = 0] remaining members to acquire global barrier 2023-02-11 03:54:19,417 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testScannerOnBulkLoadRefHFiles' starting 'in-barrier' execution. 2023-02-11 03:54:19,417 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-02-11 03:54:19,418 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-02-11 03:54:19,418 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-02-11 03:54:19,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-02-11 03:54:19,418 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testScannerOnBulkLoadRefHFiles' received 'reached' from coordinator. 2023-02-11 03:54:19,418 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testScannerOnBulkLoadRefHFiles' locally completed 2023-02-11 03:54:19,418 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:19,418 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 03:54:19,418 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testScannerOnBulkLoadRefHFiles' completed for member 'jenkins-hbase4.apache.org,37011,1676087618428' in zk 2023-02-11 03:54:19,420 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:19,420 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testScannerOnBulkLoadRefHFiles' has notified controller of completion 2023-02-11 03:54:19,420 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:19,420 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:54:19,420 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:54:19,420 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:54:19,420 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testScannerOnBulkLoadRefHFiles' completed. 2023-02-11 03:54:19,420 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:54:19,421 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:54:19,421 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-02-11 03:54:19,421 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:19,421 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:54:19,421 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-02-11 03:54:19,422 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:19,422 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testScannerOnBulkLoadRefHFiles' member 'jenkins-hbase4.apache.org,37011,1676087618428': 2023-02-11 03:54:19,422 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37011,1676087618428' released barrier for procedure'testScannerOnBulkLoadRefHFiles', counting down latch. Waiting for 0 more 2023-02-11 03:54:19,422 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testScannerOnBulkLoadRefHFiles' execution completed 2023-02-11 03:54:19,422 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 03:54:19,422 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 03:54:19,422 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testScannerOnBulkLoadRefHFiles 2023-02-11 03:54:19,422 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testScannerOnBulkLoadRefHFilesincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 03:54:19,424 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-02-11 03:54:19,424 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-02-11 03:54:19,424 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-02-11 03:54:19,424 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:54:19,424 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:54:19,424 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-02-11 03:54:19,424 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:19,424 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-02-11 03:54:19,424 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:54:19,424 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:19,424 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:19,424 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:19,424 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-02-11 03:54:19,425 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-02-11 03:54:19,425 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:54:19,425 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-02-11 03:54:19,426 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:19,426 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:19,426 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:54:19,426 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-02-11 03:54:19,426 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:19,436 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:19,436 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:19,436 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-02-11 03:54:19,436 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:19,436 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:19,436 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:19,436 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testScannerOnBulkLoadRefHFiles' 2023-02-11 03:54:19,436 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 03:54:19,436 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:54:19,436 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-02-11 03:54:19,436 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:19,436 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:19,436 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:19,436 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-02-11 03:54:19,436 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-02-11 03:54:19,437 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-02-11 03:54:19,437 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testScannerOnBulkLoadRefHFiles'' to complete. (max 300000 ms per retry) 2023-02-11 03:54:19,437 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 03:54:19,537 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 03:54:19,538 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 03:54:19,542 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 03:54:19,543 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] procedure.ProcedureCoordinator(143): Procedure testScannerOnBulkLoadRefHFiles was in running list but was completed. Accepting new attempt. 2023-02-11 03:54:19,543 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] procedure.ProcedureCoordinator(165): Submitting procedure testScannerOnBulkLoadRefHFiles 2023-02-11 03:54:19,543 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testScannerOnBulkLoadRefHFiles' 2023-02-11 03:54:19,543 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:54:19,543 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testScannerOnBulkLoadRefHFiles' starting 'acquire' 2023-02-11 03:54:19,543 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testScannerOnBulkLoadRefHFiles', kicking off acquire phase on members. 2023-02-11 03:54:19,544 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-02-11 03:54:19,544 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-02-11 03:54:19,546 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:19,546 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:19,547 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:19,547 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:19,547 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:19,547 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 03:54:19,547 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-02-11 03:54:19,547 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-02-11 03:54:19,547 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 03:54:19,548 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-02-11 03:54:19,548 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testScannerOnBulkLoadRefHFiles 2023-02-11 03:54:19,548 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testScannerOnBulkLoadRefHFiles is still present, removing 2023-02-11 03:54:19,548 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testScannerOnBulkLoadRefHFiles 2023-02-11 03:54:19,548 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testScannerOnBulkLoadRefHFiles' with timeout 60000ms 2023-02-11 03:54:19,548 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:54:19,548 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testScannerOnBulkLoadRefHFiles' starting 'acquire' stage 2023-02-11 03:54:19,548 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testScannerOnBulkLoadRefHFiles,,1676087656456.1e1751a139bc311156cd44006ffe3506. 2023-02-11 03:54:19,548 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testScannerOnBulkLoadRefHFiles,,1676087656456.1e1751a139bc311156cd44006ffe3506. started... 2023-02-11 03:54:19,548 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testScannerOnBulkLoadRefHFiles,2,1676087656456.af8a134fc8a9b827f534b0bd6e36ea9a. 2023-02-11 03:54:19,548 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-02-11 03:54:19,548 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 1e1751a139bc311156cd44006ffe3506: 2023-02-11 03:54:19,548 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 03:54:19,548 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testScannerOnBulkLoadRefHFiles,1,1676087656456.2ed5c51addf313895c503de93f35da1a. 2023-02-11 03:54:19,548 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testScannerOnBulkLoadRefHFiles,1,1676087656456.2ed5c51addf313895c503de93f35da1a. started... 2023-02-11 03:54:19,548 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testScannerOnBulkLoadRefHFiles,2,1676087656456.af8a134fc8a9b827f534b0bd6e36ea9a. started... 2023-02-11 03:54:19,548 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testScannerOnBulkLoadRefHFiles,,1676087656456.1e1751a139bc311156cd44006ffe3506. 2023-02-11 03:54:19,549 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-2] regionserver.HRegion(2445): Flush status journal for 2ed5c51addf313895c503de93f35da1a: 2023-02-11 03:54:19,549 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-02-11 03:54:19,549 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-1] regionserver.HRegion(2744): Flushing af8a134fc8a9b827f534b0bd6e36ea9a 2/2 column families, dataSize=90 B heapSize=752 B 2023-02-11 03:54:19,549 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testScannerOnBulkLoadRefHFiles,1,1676087656456.2ed5c51addf313895c503de93f35da1a. 2023-02-11 03:54:19,549 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-02-11 03:54:19,963 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230211b0d290ca9a0b424297a5b847260ead85 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testScannerOnBulkLoadRefHFiles/9a980ee3ec24e1d8bfa59c2e531fd315/family1/c81e728d9d4c2f636f067f89cc14862c20230211b0d290ca9a0b424297a5b847260ead85 2023-02-11 03:54:19,969 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=90, hasBloomFilter=true, into tmp file hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testScannerOnBulkLoadRefHFiles/af8a134fc8a9b827f534b0bd6e36ea9a/.tmp/family1/cc4acceacc7f4bf7a232a73fd9645aa6 2023-02-11 03:54:19,977 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testScannerOnBulkLoadRefHFiles/af8a134fc8a9b827f534b0bd6e36ea9a/.tmp/family1/cc4acceacc7f4bf7a232a73fd9645aa6 as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testScannerOnBulkLoadRefHFiles/af8a134fc8a9b827f534b0bd6e36ea9a/family1/cc4acceacc7f4bf7a232a73fd9645aa6 2023-02-11 03:54:19,982 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testScannerOnBulkLoadRefHFiles/af8a134fc8a9b827f534b0bd6e36ea9a/family1/cc4acceacc7f4bf7a232a73fd9645aa6, entries=2, sequenceid=9, filesize=5.0 K 2023-02-11 03:54:19,984 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-1] regionserver.HRegion(2947): Finished flush of dataSize ~90 B/90, heapSize ~480 B/480, currentSize=0 B/0 for af8a134fc8a9b827f534b0bd6e36ea9a in 434ms, sequenceid=9, compaction requested=false 2023-02-11 03:54:19,984 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-1] regionserver.HRegion(2445): Flush status journal for af8a134fc8a9b827f534b0bd6e36ea9a: 2023-02-11 03:54:19,984 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testScannerOnBulkLoadRefHFiles,2,1676087656456.af8a134fc8a9b827f534b0bd6e36ea9a. 2023-02-11 03:54:19,985 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-02-11 03:54:19,985 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-02-11 03:54:19,985 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:19,985 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testScannerOnBulkLoadRefHFiles' locally acquired 2023-02-11 03:54:19,985 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37011,1676087618428' joining acquired barrier for procedure (testScannerOnBulkLoadRefHFiles) in zk 2023-02-11 03:54:19,986 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-02-11 03:54:19,986 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:19,986 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:19,987 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:54:19,987 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:54:19,987 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-02-11 03:54:19,987 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testScannerOnBulkLoadRefHFiles' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 03:54:19,987 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:54:19,987 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:54:19,987 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-02-11 03:54:19,988 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:19,988 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:54:19,988 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37011,1676087618428' joining acquired barrier for procedure 'testScannerOnBulkLoadRefHFiles' on coordinator 2023-02-11 03:54:19,988 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1a857cb2[Count = 0] remaining members to acquire global barrier 2023-02-11 03:54:19,988 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testScannerOnBulkLoadRefHFiles' starting 'in-barrier' execution. 2023-02-11 03:54:19,989 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-02-11 03:54:19,990 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-02-11 03:54:19,990 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-02-11 03:54:19,990 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-02-11 03:54:19,990 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testScannerOnBulkLoadRefHFiles' received 'reached' from coordinator. 2023-02-11 03:54:19,990 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testScannerOnBulkLoadRefHFiles' locally completed 2023-02-11 03:54:19,990 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testScannerOnBulkLoadRefHFiles' completed for member 'jenkins-hbase4.apache.org,37011,1676087618428' in zk 2023-02-11 03:54:19,990 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:19,990 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 03:54:19,993 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:19,993 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testScannerOnBulkLoadRefHFiles' has notified controller of completion 2023-02-11 03:54:19,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:19,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:54:19,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:54:19,993 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:54:19,993 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testScannerOnBulkLoadRefHFiles' completed. 2023-02-11 03:54:20,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:54:20,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:54:20,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-02-11 03:54:20,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:20,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:54:20,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-02-11 03:54:20,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:20,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testScannerOnBulkLoadRefHFiles' member 'jenkins-hbase4.apache.org,37011,1676087618428': 2023-02-11 03:54:20,003 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37011,1676087618428' released barrier for procedure'testScannerOnBulkLoadRefHFiles', counting down latch. Waiting for 0 more 2023-02-11 03:54:20,003 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testScannerOnBulkLoadRefHFiles' execution completed 2023-02-11 03:54:20,003 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 03:54:20,003 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 03:54:20,003 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testScannerOnBulkLoadRefHFiles 2023-02-11 03:54:20,003 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testScannerOnBulkLoadRefHFilesincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 03:54:20,006 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-02-11 03:54:20,006 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:20,006 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-02-11 03:54:20,006 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-02-11 03:54:20,006 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-02-11 03:54:20,006 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-02-11 03:54:20,006 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:54:20,006 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:54:20,006 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:20,006 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:20,006 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:54:20,007 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:20,007 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-02-11 03:54:20,007 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-02-11 03:54:20,007 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:54:20,007 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-02-11 03:54:20,008 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:20,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:20,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:54:20,009 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-02-11 03:54:20,009 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:20,010 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:20,010 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:20,011 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:54:20,011 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles because node does not exist (not an error) 2023-02-11 03:54:20,011 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-02-11 03:54:20,011 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:20,011 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:20,011 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:20,011 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-02-11 03:54:20,011 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testScannerOnBulkLoadRefHFiles' 2023-02-11 03:54:20,011 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 03:54:20,011 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:20,011 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-02-11 03:54:20,011 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-02-11 03:54:20,011 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-02-11 03:54:20,011 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testScannerOnBulkLoadRefHFiles'' to complete. (max 300000 ms per retry) 2023-02-11 03:54:20,012 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:20,012 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:20,012 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 03:54:20,112 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 03:54:20,113 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 03:54:20,119 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=39921] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}] of the table testScannerOnBulkLoadRefHFiles 2023-02-11 03:54:20,121 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676087621935] procedure2.ProcedureExecutor(1029): Stored pid=34, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testScannerOnBulkLoadRefHFiles.mobLock, type=EXCLUSIVE 2023-02-11 03:54:20,122 DEBUG [PEWorker-5] locking.LockProcedure(309): LOCKED pid=34, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testScannerOnBulkLoadRefHFiles.mobLock, type=EXCLUSIVE 2023-02-11 03:54:20,123 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676087621935] mob.MobUtils(833): start MOB compaction of files for table='testScannerOnBulkLoadRefHFiles', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-11 03:54:20,123 INFO [PEWorker-5] procedure2.TimeoutExecutorThread(81): ADDED pid=34, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testScannerOnBulkLoadRefHFiles.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1676088260123 2023-02-11 03:54:20,123 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676087621935] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-02-11 03:54:20,124 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676087621935] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 2 selected files, and 0 irrelevant files table 'testScannerOnBulkLoadRefHFiles' and column 'family1' 2023-02-11 03:54:20,124 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676087621935] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testScannerOnBulkLoadRefHFiles' column='family1' 2023-02-11 03:54:20,124 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676087621935] zookeeper.ReadOnlyZKClient(139): Connect 0x307d34e9 to 127.0.0.1:53383 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-11 03:54:20,128 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676087621935] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@2e349a3, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-11 03:54:20,128 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c81e728d9d4c2f636f067f89cc14862c20230211 for table='testScannerOnBulkLoadRefHFiles' and column='family1' 2023-02-11 03:54:20,960 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c2023021103f84e07d61d4c8988af5829d7300c6e to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testScannerOnBulkLoadRefHFiles/9a980ee3ec24e1d8bfa59c2e531fd315/family1/c81e728d9d4c2f636f067f89cc14862c2023021103f84e07d61d4c8988af5829d7300c6e 2023-02-11 03:54:20,961 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20230211 table='testScannerOnBulkLoadRefHFiles' column='family1' 2023-02-11 03:54:20,973 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-11 03:54:20,975 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:44974, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-11 03:54:20,981 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-02-11 03:54:20,983 INFO [RS-EventLoopGroup-1-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:36376, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-02-11 03:54:20,991 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/.bulkload/default/testScannerOnBulkLoadRefHFiles/c81e728d9d4c2f636f067f89cc14862c20230211/family1/c8e2435adeb14188850ad0407c4f9121 first=Optional[k0] last=Optional[k1] 2023-02-11 03:54:20,992 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testScannerOnBulkLoadRefHFiles,2,1676087656456.af8a134fc8a9b827f534b0bd6e36ea9a., hostname=jenkins-hbase4.apache.org,37011,1676087618428, seqNum=2 for row 2 with hfile group [{family1,hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/.bulkload/default/testScannerOnBulkLoadRefHFiles/c81e728d9d4c2f636f067f89cc14862c20230211/family1/c8e2435adeb14188850ad0407c4f9121}] 2023-02-11 03:54:20,995 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.HStore(763): Validating hfile at hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/.bulkload/default/testScannerOnBulkLoadRefHFiles/c81e728d9d4c2f636f067f89cc14862c20230211/family1/c8e2435adeb14188850ad0407c4f9121 for inclusion in af8a134fc8a9b827f534b0bd6e36ea9a/family1 2023-02-11 03:54:20,998 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.HStore(775): HFile bounds: first=k0 last=k1 2023-02-11 03:54:20,998 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.HStore(777): Region bounds: first=2 last= 2023-02-11 03:54:20,998 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.HRegion(2445): Flush status journal for af8a134fc8a9b827f534b0bd6e36ea9a: 2023-02-11 03:54:20,999 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/.bulkload/default/testScannerOnBulkLoadRefHFiles/c81e728d9d4c2f636f067f89cc14862c20230211/family1/c8e2435adeb14188850ad0407c4f9121 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/staging/jenkins__testScannerOnBulkLoadRefHFiles__a282ki2pk0pp4igom2au8qr70t7tp7nkbh4en2v792c9u95acc17t08r02qh2qtm/family1/c8e2435adeb14188850ad0407c4f9121 2023-02-11 03:54:21,001 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/staging/jenkins__testScannerOnBulkLoadRefHFiles__a282ki2pk0pp4igom2au8qr70t7tp7nkbh4en2v792c9u95acc17t08r02qh2qtm/family1/c8e2435adeb14188850ad0407c4f9121 as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testScannerOnBulkLoadRefHFiles/af8a134fc8a9b827f534b0bd6e36ea9a/family1/b512b95e5591486f97cae13833a6f90c_SeqId_12_ 2023-02-11 03:54:21,001 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.HStore(849): Loaded HFile hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/staging/jenkins__testScannerOnBulkLoadRefHFiles__a282ki2pk0pp4igom2au8qr70t7tp7nkbh4en2v792c9u95acc17t08r02qh2qtm/family1/c8e2435adeb14188850ad0407c4f9121 into af8a134fc8a9b827f534b0bd6e36ea9a/family1 as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testScannerOnBulkLoadRefHFiles/af8a134fc8a9b827f534b0bd6e36ea9a/family1/b512b95e5591486f97cae13833a6f90c_SeqId_12_ - updating store file list. 2023-02-11 03:54:21,005 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.HStore(882): Loaded HFile hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testScannerOnBulkLoadRefHFiles/af8a134fc8a9b827f534b0bd6e36ea9a/family1/b512b95e5591486f97cae13833a6f90c_SeqId_12_ into af8a134fc8a9b827f534b0bd6e36ea9a/family1 2023-02-11 03:54:21,005 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.HStore(855): Successfully loaded hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/staging/jenkins__testScannerOnBulkLoadRefHFiles__a282ki2pk0pp4igom2au8qr70t7tp7nkbh4en2v792c9u95acc17t08r02qh2qtm/family1/c8e2435adeb14188850ad0407c4f9121 into af8a134fc8a9b827f534b0bd6e36ea9a/family1 (new location: hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testScannerOnBulkLoadRefHFiles/af8a134fc8a9b827f534b0bd6e36ea9a/family1/b512b95e5591486f97cae13833a6f90c_SeqId_12_) 2023-02-11 03:54:21,005 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/staging/jenkins__testScannerOnBulkLoadRefHFiles__a282ki2pk0pp4igom2au8qr70t7tp7nkbh4en2v792c9u95acc17t08r02qh2qtm/family1/c8e2435adeb14188850ad0407c4f9121 2023-02-11 03:54:21,005 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-02-11 03:54:21,006 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 15445 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-02-11 03:54:21,006 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.HStore(1912): af8a134fc8a9b827f534b0bd6e36ea9a/family1 is initiating minor compaction (all files) 2023-02-11 03:54:21,006 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@1e0d8aef; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-11 03:54:21,006 INFO [RS:0;jenkins-hbase4:37011-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of af8a134fc8a9b827f534b0bd6e36ea9a/family1 in testScannerOnBulkLoadRefHFiles,2,1676087656456.af8a134fc8a9b827f534b0bd6e36ea9a. 2023-02-11 03:54:21,007 INFO [RS:0;jenkins-hbase4:37011-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testScannerOnBulkLoadRefHFiles/af8a134fc8a9b827f534b0bd6e36ea9a/family1/a771c94e53fe4dc3adb4c25c4f94cad0, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testScannerOnBulkLoadRefHFiles/af8a134fc8a9b827f534b0bd6e36ea9a/family1/cc4acceacc7f4bf7a232a73fd9645aa6, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testScannerOnBulkLoadRefHFiles/af8a134fc8a9b827f534b0bd6e36ea9a/family1/b512b95e5591486f97cae13833a6f90c_SeqId_12_] into tmpdir=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testScannerOnBulkLoadRefHFiles/af8a134fc8a9b827f534b0bd6e36ea9a/.tmp, totalSize=15.1 K 2023-02-11 03:54:21,007 DEBUG [RS:0;jenkins-hbase4:37011-shortCompactions-0] compactions.Compactor(207): Compacting a771c94e53fe4dc3adb4c25c4f94cad0, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1676087658566 2023-02-11 03:54:21,008 DEBUG [RS:0;jenkins-hbase4:37011-shortCompactions-0] compactions.Compactor(207): Compacting cc4acceacc7f4bf7a232a73fd9645aa6, keycount=2, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1676087658566 2023-02-11 03:54:21,008 DEBUG [RS:0;jenkins-hbase4:37011-shortCompactions-0] compactions.Compactor(207): Compacting b512b95e5591486f97cae13833a6f90c_SeqId_12_, keycount=2, bloomtype=ROW, size=5.2 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1676087658566 2023-02-11 03:54:21,008 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20230211 table='testScannerOnBulkLoadRefHFiles' column='family1' 2023-02-11 03:54:21,009 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-11 03:54:21,011 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testScannerOnBulkLoadRefHFiles/9a980ee3ec24e1d8bfa59c2e531fd315/family1/c81e728d9d4c2f636f067f89cc14862c20230211b0d290ca9a0b424297a5b847260ead85 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testScannerOnBulkLoadRefHFiles/9a980ee3ec24e1d8bfa59c2e531fd315/family1/c81e728d9d4c2f636f067f89cc14862c20230211b0d290ca9a0b424297a5b847260ead85 2023-02-11 03:54:21,012 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testScannerOnBulkLoadRefHFiles/9a980ee3ec24e1d8bfa59c2e531fd315/family1/c81e728d9d4c2f636f067f89cc14862c20230211db356e8ae10b496fbd00b289cbd1c3e0 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testScannerOnBulkLoadRefHFiles/9a980ee3ec24e1d8bfa59c2e531fd315/family1/c81e728d9d4c2f636f067f89cc14862c20230211db356e8ae10b496fbd00b289cbd1c3e0 2023-02-11 03:54:21,012 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testScannerOnBulkLoadRefHFiles 2 expired mob files are deleted 2023-02-11 03:54:21,012 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/.bulkload/default/testScannerOnBulkLoadRefHFiles/c81e728d9d4c2f636f067f89cc14862c20230211' 2023-02-11 03:54:21,012 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 2 to 1 for partition=c81e728d9d4c2f636f067f89cc14862c20230211 for table='testScannerOnBulkLoadRefHFiles' and column='family1' 2023-02-11 03:54:21,013 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676087621935] compactions.PartitionedMobCompactor(354): After compaction, there are 1 mob files. table='testScannerOnBulkLoadRefHFiles' column='family1' 2023-02-11 03:54:21,013 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676087621935] mob.MobUtils(840): end MOB compaction of files for table='testScannerOnBulkLoadRefHFiles', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-11 03:54:21,014 DEBUG [PEWorker-2] locking.LockProcedure(242): UNLOCKED pid=34, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testScannerOnBulkLoadRefHFiles.mobLock, type=EXCLUSIVE 2023-02-11 03:54:21,015 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=34, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testScannerOnBulkLoadRefHFiles.mobLock, type=EXCLUSIVE in 894 msec 2023-02-11 03:54:21,018 INFO [RS:0;jenkins-hbase4:37011-shortCompactions-0] throttle.PressureAwareThroughputController(145): af8a134fc8a9b827f534b0bd6e36ea9a#family1#compaction#38 average throughput is 0.06 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-11 03:54:21,046 INFO [Listener at localhost/46205] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testScannerOnBulkLoadRefHFiles Thread=255 (was 256), OpenFileDescriptor=609 (was 608) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=231 (was 231), ProcessCount=168 (was 168), AvailableMemoryMB=3291 (was 3400) 2023-02-11 03:54:21,056 INFO [Listener at localhost/46205] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMajorCompactionWithWeeklyPolicy Thread=255, OpenFileDescriptor=609, MaxFileDescriptor=60000, SystemLoadAverage=231, ProcessCount=168, AvailableMemoryMB=3290 2023-02-11 03:54:21,058 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] master.HMaster$4(2102): Client=jenkins//172.31.14.131 create 'testMajorCompactionWithWeeklyPolicy', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} 2023-02-11 03:54:21,059 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] procedure2.ProcedureExecutor(1029): Stored pid=35, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:21,060 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=35, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_PRE_OPERATION 2023-02-11 03:54:21,061 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] master.MasterRpcServices(697): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testMajorCompactionWithWeeklyPolicy" procId is: 35 2023-02-11 03:54:21,061 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=35, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-11 03:54:21,061 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] master.MasterRpcServices(1227): Checking to see if procedure is done pid=35 2023-02-11 03:54:21,062 DEBUG [HFileArchiver-8] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/.tmp/data/default/testMajorCompactionWithWeeklyPolicy/cb6565dc6679ffd9e191e6bacbcc214d 2023-02-11 03:54:21,063 DEBUG [HFileArchiver-8] backup.HFileArchiver(153): Directory hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/.tmp/data/default/testMajorCompactionWithWeeklyPolicy/cb6565dc6679ffd9e191e6bacbcc214d empty. 2023-02-11 03:54:21,063 DEBUG [HFileArchiver-8] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/.tmp/data/default/testMajorCompactionWithWeeklyPolicy/cb6565dc6679ffd9e191e6bacbcc214d 2023-02-11 03:54:21,063 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testMajorCompactionWithWeeklyPolicy regions 2023-02-11 03:54:21,162 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] master.MasterRpcServices(1227): Checking to see if procedure is done pid=35 2023-02-11 03:54:21,363 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] master.MasterRpcServices(1227): Checking to see if procedure is done pid=35 2023-02-11 03:54:21,471 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/.tmp/data/default/testMajorCompactionWithWeeklyPolicy/.tabledesc/.tableinfo.0000000001 2023-02-11 03:54:21,472 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(7671): creating {ENCODED => cb6565dc6679ffd9e191e6bacbcc214d, NAME => 'testMajorCompactionWithWeeklyPolicy,,1676087661058.cb6565dc6679ffd9e191e6bacbcc214d.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testMajorCompactionWithWeeklyPolicy', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}, regionDir=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/.tmp 2023-02-11 03:54:21,664 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] master.MasterRpcServices(1227): Checking to see if procedure is done pid=35 2023-02-11 03:54:21,878 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(865): Instantiated testMajorCompactionWithWeeklyPolicy,,1676087661058.cb6565dc6679ffd9e191e6bacbcc214d.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-11 03:54:21,878 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1603): Closing cb6565dc6679ffd9e191e6bacbcc214d, disabling compactions & flushes 2023-02-11 03:54:21,878 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1625): Closing region testMajorCompactionWithWeeklyPolicy,,1676087661058.cb6565dc6679ffd9e191e6bacbcc214d. 2023-02-11 03:54:21,878 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testMajorCompactionWithWeeklyPolicy,,1676087661058.cb6565dc6679ffd9e191e6bacbcc214d. 2023-02-11 03:54:21,878 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1713): Acquired close lock on testMajorCompactionWithWeeklyPolicy,,1676087661058.cb6565dc6679ffd9e191e6bacbcc214d. after waiting 0 ms 2023-02-11 03:54:21,878 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1723): Updates disabled for region testMajorCompactionWithWeeklyPolicy,,1676087661058.cb6565dc6679ffd9e191e6bacbcc214d. 2023-02-11 03:54:21,878 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1837): Closed testMajorCompactionWithWeeklyPolicy,,1676087661058.cb6565dc6679ffd9e191e6bacbcc214d. 2023-02-11 03:54:21,878 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1557): Region close journal for cb6565dc6679ffd9e191e6bacbcc214d: Waiting for close lock at 1676087661878Disabling compacts and flushes for region at 1676087661878Disabling writes for close at 1676087661878Writing region close event to WAL at 1676087661878Closed at 1676087661878 2023-02-11 03:54:21,881 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=35, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_ADD_TO_META 2023-02-11 03:54:21,881 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyPolicy,,1676087661058.cb6565dc6679ffd9e191e6bacbcc214d.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1676087661881"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676087661881"}]},"ts":"1676087661881"} 2023-02-11 03:54:21,883 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-11 03:54:21,884 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=35, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-11 03:54:21,884 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithWeeklyPolicy","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676087661884"}]},"ts":"1676087661884"} 2023-02-11 03:54:21,885 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithWeeklyPolicy, state=ENABLING in hbase:meta 2023-02-11 03:54:21,891 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=36, ppid=35, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyPolicy, region=cb6565dc6679ffd9e191e6bacbcc214d, ASSIGN}] 2023-02-11 03:54:21,892 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=36, ppid=35, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyPolicy, region=cb6565dc6679ffd9e191e6bacbcc214d, ASSIGN 2023-02-11 03:54:21,893 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=36, ppid=35, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyPolicy, region=cb6565dc6679ffd9e191e6bacbcc214d, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,37011,1676087618428; forceNewPlan=false, retain=false 2023-02-11 03:54:22,044 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=36 updating hbase:meta row=cb6565dc6679ffd9e191e6bacbcc214d, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:22,044 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyPolicy,,1676087661058.cb6565dc6679ffd9e191e6bacbcc214d.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1676087662044"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676087662044"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676087662044"}]},"ts":"1676087662044"} 2023-02-11 03:54:22,046 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=37, ppid=36, state=RUNNABLE; OpenRegionProcedure cb6565dc6679ffd9e191e6bacbcc214d, server=jenkins-hbase4.apache.org,37011,1676087618428}] 2023-02-11 03:54:22,165 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] master.MasterRpcServices(1227): Checking to see if procedure is done pid=35 2023-02-11 03:54:22,200 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionWithWeeklyPolicy,,1676087661058.cb6565dc6679ffd9e191e6bacbcc214d. 2023-02-11 03:54:22,200 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => cb6565dc6679ffd9e191e6bacbcc214d, NAME => 'testMajorCompactionWithWeeklyPolicy,,1676087661058.cb6565dc6679ffd9e191e6bacbcc214d.', STARTKEY => '', ENDKEY => ''} 2023-02-11 03:54:22,200 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyPolicy cb6565dc6679ffd9e191e6bacbcc214d 2023-02-11 03:54:22,200 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testMajorCompactionWithWeeklyPolicy,,1676087661058.cb6565dc6679ffd9e191e6bacbcc214d.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-11 03:54:22,200 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for cb6565dc6679ffd9e191e6bacbcc214d 2023-02-11 03:54:22,201 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for cb6565dc6679ffd9e191e6bacbcc214d 2023-02-11 03:54:22,202 INFO [StoreOpener-cb6565dc6679ffd9e191e6bacbcc214d-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region cb6565dc6679ffd9e191e6bacbcc214d 2023-02-11 03:54:22,203 DEBUG [StoreOpener-cb6565dc6679ffd9e191e6bacbcc214d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyPolicy/cb6565dc6679ffd9e191e6bacbcc214d/family1 2023-02-11 03:54:22,203 DEBUG [StoreOpener-cb6565dc6679ffd9e191e6bacbcc214d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyPolicy/cb6565dc6679ffd9e191e6bacbcc214d/family1 2023-02-11 03:54:22,204 INFO [StoreOpener-cb6565dc6679ffd9e191e6bacbcc214d-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region cb6565dc6679ffd9e191e6bacbcc214d columnFamilyName family1 2023-02-11 03:54:22,205 INFO [StoreOpener-cb6565dc6679ffd9e191e6bacbcc214d-1] regionserver.HStore(310): Store=cb6565dc6679ffd9e191e6bacbcc214d/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 03:54:22,206 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyPolicy/cb6565dc6679ffd9e191e6bacbcc214d 2023-02-11 03:54:22,207 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyPolicy/cb6565dc6679ffd9e191e6bacbcc214d 2023-02-11 03:54:22,209 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for cb6565dc6679ffd9e191e6bacbcc214d 2023-02-11 03:54:22,211 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyPolicy/cb6565dc6679ffd9e191e6bacbcc214d/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-11 03:54:22,212 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened cb6565dc6679ffd9e191e6bacbcc214d; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=72726323, jitterRate=0.08370666205883026}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-11 03:54:22,212 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for cb6565dc6679ffd9e191e6bacbcc214d: Running coprocessor pre-open hook at 1676087662201Writing region info on filesystem at 1676087662201Initializing all the Stores at 1676087662201Instantiating store for column family {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} at 1676087662201Cleaning up temporary data from old regions at 1676087662207 (+6 ms)Cleaning up detritus from prior splits at 1676087662207Running coprocessor post-open hooks at 1676087662212 (+5 ms)Region opened successfully at 1676087662212 2023-02-11 03:54:22,212 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionWithWeeklyPolicy,,1676087661058.cb6565dc6679ffd9e191e6bacbcc214d., pid=37, masterSystemTime=1676087662197 2023-02-11 03:54:22,214 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionWithWeeklyPolicy,,1676087661058.cb6565dc6679ffd9e191e6bacbcc214d. 2023-02-11 03:54:22,214 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionWithWeeklyPolicy,,1676087661058.cb6565dc6679ffd9e191e6bacbcc214d. 2023-02-11 03:54:22,214 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=36 updating hbase:meta row=cb6565dc6679ffd9e191e6bacbcc214d, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:22,214 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyPolicy,,1676087661058.cb6565dc6679ffd9e191e6bacbcc214d.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1676087662214"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676087662214"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676087662214"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676087662214"}]},"ts":"1676087662214"} 2023-02-11 03:54:22,217 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=37, resume processing ppid=36 2023-02-11 03:54:22,217 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=37, ppid=36, state=SUCCESS; OpenRegionProcedure cb6565dc6679ffd9e191e6bacbcc214d, server=jenkins-hbase4.apache.org,37011,1676087618428 in 171 msec 2023-02-11 03:54:22,219 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=36, resume processing ppid=35 2023-02-11 03:54:22,219 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=36, ppid=35, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyPolicy, region=cb6565dc6679ffd9e191e6bacbcc214d, ASSIGN in 327 msec 2023-02-11 03:54:22,220 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=35, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-11 03:54:22,220 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithWeeklyPolicy","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676087662220"}]},"ts":"1676087662220"} 2023-02-11 03:54:22,221 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithWeeklyPolicy, state=ENABLED in hbase:meta 2023-02-11 03:54:22,223 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=35, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_POST_OPERATION 2023-02-11 03:54:22,225 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=35, state=SUCCESS; CreateTableProcedure table=testMajorCompactionWithWeeklyPolicy in 1.1650 sec 2023-02-11 03:54:22,238 DEBUG [RS:0;jenkins-hbase4:37011-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testScannerOnBulkLoadRefHFiles/af8a134fc8a9b827f534b0bd6e36ea9a/.tmp/family1/d6ff4b1b45ba4e46af50a882975a592a as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testScannerOnBulkLoadRefHFiles/af8a134fc8a9b827f534b0bd6e36ea9a/family1/d6ff4b1b45ba4e46af50a882975a592a 2023-02-11 03:54:22,243 INFO [RS:0;jenkins-hbase4:37011-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in af8a134fc8a9b827f534b0bd6e36ea9a/family1 of af8a134fc8a9b827f534b0bd6e36ea9a into d6ff4b1b45ba4e46af50a882975a592a(size=5.1 K), total size for store is 5.1 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-02-11 03:54:22,243 DEBUG [RS:0;jenkins-hbase4:37011-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for af8a134fc8a9b827f534b0bd6e36ea9a: 2023-02-11 03:54:22,243 INFO [RS:0;jenkins-hbase4:37011-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testScannerOnBulkLoadRefHFiles,2,1676087656456.af8a134fc8a9b827f534b0bd6e36ea9a., storeName=family1, fileCount=3, fileSize=15.1 K (4.9 K, 5.0 K, 5.2 K), priority=2, time=1676087661006; duration=1sec 2023-02-11 03:54:22,243 DEBUG [RS:0;jenkins-hbase4:37011-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-11 03:54:22,288 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testScannerOnBulkLoadRefHFiles/af8a134fc8a9b827f534b0bd6e36ea9a/family1/a771c94e53fe4dc3adb4c25c4f94cad0, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testScannerOnBulkLoadRefHFiles/af8a134fc8a9b827f534b0bd6e36ea9a/family1/cc4acceacc7f4bf7a232a73fd9645aa6, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testScannerOnBulkLoadRefHFiles/af8a134fc8a9b827f534b0bd6e36ea9a/family1/b512b95e5591486f97cae13833a6f90c_SeqId_12_] to archive 2023-02-11 03:54:22,289 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-11 03:54:22,291 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testScannerOnBulkLoadRefHFiles/af8a134fc8a9b827f534b0bd6e36ea9a/family1/a771c94e53fe4dc3adb4c25c4f94cad0 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testScannerOnBulkLoadRefHFiles/af8a134fc8a9b827f534b0bd6e36ea9a/family1/a771c94e53fe4dc3adb4c25c4f94cad0 2023-02-11 03:54:22,292 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testScannerOnBulkLoadRefHFiles/af8a134fc8a9b827f534b0bd6e36ea9a/family1/cc4acceacc7f4bf7a232a73fd9645aa6 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testScannerOnBulkLoadRefHFiles/af8a134fc8a9b827f534b0bd6e36ea9a/family1/cc4acceacc7f4bf7a232a73fd9645aa6 2023-02-11 03:54:22,293 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testScannerOnBulkLoadRefHFiles/af8a134fc8a9b827f534b0bd6e36ea9a/family1/b512b95e5591486f97cae13833a6f90c_SeqId_12_ to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testScannerOnBulkLoadRefHFiles/af8a134fc8a9b827f534b0bd6e36ea9a/family1/b512b95e5591486f97cae13833a6f90c_SeqId_12_ 2023-02-11 03:54:23,134 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-11 03:54:23,166 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] master.MasterRpcServices(1227): Checking to see if procedure is done pid=35 2023-02-11 03:54:23,166 INFO [Listener at localhost/46205] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testMajorCompactionWithWeeklyPolicy, procId: 35 completed 2023-02-11 03:54:23,200 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 03:54:23,208 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:23,208 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-02-11 03:54:23,208 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:54:23,208 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-02-11 03:54:23,208 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-02-11 03:54:23,209 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:23,209 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:23,210 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:23,210 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:23,210 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:23,210 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:23,210 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:23,210 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:23,210 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 03:54:23,211 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:23,211 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 03:54:23,211 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:23,211 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:23,212 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:23,212 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-02-11 03:54:23,212 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:54:23,212 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-02-11 03:54:23,215 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 03:54:23,216 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 03:54:23,216 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1676087661058.cb6565dc6679ffd9e191e6bacbcc214d. 2023-02-11 03:54:23,216 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1676087661058.cb6565dc6679ffd9e191e6bacbcc214d. started... 2023-02-11 03:54:23,217 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing cb6565dc6679ffd9e191e6bacbcc214d 1/1 column families, dataSize=74.11 KB heapSize=148.69 KB 2023-02-11 03:54:23,248 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151130bbcd2930eb684aab800d930970b5e2b1 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151130bbcd2930eb684aab800d930970b5e2b1 2023-02-11 03:54:23,251 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=74.1 K, hasBloomFilter=true, into tmp file hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyPolicy/cb6565dc6679ffd9e191e6bacbcc214d/.tmp/family1/5af56c38f14e43f4a62cdb0dd1942bd0 2023-02-11 03:54:23,258 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyPolicy/cb6565dc6679ffd9e191e6bacbcc214d/.tmp/family1/5af56c38f14e43f4a62cdb0dd1942bd0 as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyPolicy/cb6565dc6679ffd9e191e6bacbcc214d/family1/5af56c38f14e43f4a62cdb0dd1942bd0 2023-02-11 03:54:23,264 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyPolicy/cb6565dc6679ffd9e191e6bacbcc214d/family1/5af56c38f14e43f4a62cdb0dd1942bd0, entries=1000, sequenceid=5, filesize=165.1 K 2023-02-11 03:54:23,265 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~74.11 KB/75890, heapSize ~148.67 KB/152240, currentSize=0 B/0 for cb6565dc6679ffd9e191e6bacbcc214d in 48ms, sequenceid=5, compaction requested=false 2023-02-11 03:54:23,265 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testMajorCompactionWithWeeklyPolicy' 2023-02-11 03:54:23,266 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for cb6565dc6679ffd9e191e6bacbcc214d: 2023-02-11 03:54:23,266 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1676087661058.cb6565dc6679ffd9e191e6bacbcc214d. 2023-02-11 03:54:23,266 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 03:54:23,266 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 03:54:23,266 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:23,266 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-02-11 03:54:23,266 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37011,1676087618428' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-02-11 03:54:23,268 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:23,268 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:23,268 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:23,268 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:54:23,268 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:54:23,268 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:23,268 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:54:23,268 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 03:54:23,269 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:54:23,269 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:23,269 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:23,269 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:54:23,269 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37011,1676087618428' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-02-11 03:54:23,270 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2b6ad846[Count = 0] remaining members to acquire global barrier 2023-02-11 03:54:23,270 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-02-11 03:54:23,270 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:23,272 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:23,272 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:23,272 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:23,272 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-02-11 03:54:23,272 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-02-11 03:54:23,272 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,37011,1676087618428' in zk 2023-02-11 03:54:23,272 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:23,272 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 03:54:23,273 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:23,274 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:23,274 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:54:23,274 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:54:23,274 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-02-11 03:54:23,274 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:54:23,274 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-02-11 03:54:23,275 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:54:23,276 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:54:23,276 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:23,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:23,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:54:23,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:23,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:23,279 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,37011,1676087618428': 2023-02-11 03:54:23,279 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37011,1676087618428' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-02-11 03:54:23,279 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-02-11 03:54:23,279 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 03:54:23,279 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 03:54:23,279 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:23,279 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 03:54:23,281 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:23,281 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:23,282 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:23,282 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:23,282 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:23,282 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:54:23,282 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:54:23,282 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:23,282 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:23,282 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:23,282 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:54:23,282 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:23,282 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:23,282 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:23,283 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:54:23,283 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:23,283 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:23,284 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:23,284 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:54:23,285 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:23,285 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:23,287 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:23,287 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:23,287 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:23,287 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:23,287 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:23,287 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:23,287 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-02-11 03:54:23,287 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 03:54:23,287 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:23,287 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:54:23,287 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:23,287 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:23,287 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:23,287 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:23,287 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:23,288 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:23,288 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-02-11 03:54:23,288 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 03:54:23,388 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 03:54:23,389 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 03:54:23,394 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 03:54:23,395 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-02-11 03:54:23,395 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:23,395 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-02-11 03:54:23,395 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:54:23,395 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-02-11 03:54:23,395 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-02-11 03:54:23,396 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:23,396 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:23,398 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:23,398 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:23,398 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:23,398 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:23,398 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:23,398 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 03:54:23,398 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:23,398 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:23,399 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 03:54:23,399 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:23,399 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:23,399 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-02-11 03:54:23,399 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:23,399 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-02-11 03:54:23,399 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:54:23,399 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-02-11 03:54:23,399 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 03:54:23,399 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 03:54:23,399 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1676087661058.cb6565dc6679ffd9e191e6bacbcc214d. 2023-02-11 03:54:23,399 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1676087661058.cb6565dc6679ffd9e191e6bacbcc214d. started... 2023-02-11 03:54:23,399 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing cb6565dc6679ffd9e191e6bacbcc214d 1/1 column families, dataSize=74 B heapSize=408 B 2023-02-11 03:54:23,817 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151128c0fed680c2724006802f16ab2e584308 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151128c0fed680c2724006802f16ab2e584308 2023-02-11 03:54:24,224 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=74, hasBloomFilter=true, into tmp file hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyPolicy/cb6565dc6679ffd9e191e6bacbcc214d/.tmp/family1/365ad5fc2a874c7eac0ce82e29ac3375 2023-02-11 03:54:24,229 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyPolicy/cb6565dc6679ffd9e191e6bacbcc214d/.tmp/family1/365ad5fc2a874c7eac0ce82e29ac3375 as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyPolicy/cb6565dc6679ffd9e191e6bacbcc214d/family1/365ad5fc2a874c7eac0ce82e29ac3375 2023-02-11 03:54:24,234 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyPolicy/cb6565dc6679ffd9e191e6bacbcc214d/family1/365ad5fc2a874c7eac0ce82e29ac3375, entries=1, sequenceid=9, filesize=5.0 K 2023-02-11 03:54:24,234 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~74 B/74, heapSize ~392 B/392, currentSize=0 B/0 for cb6565dc6679ffd9e191e6bacbcc214d in 835ms, sequenceid=9, compaction requested=false 2023-02-11 03:54:24,235 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for cb6565dc6679ffd9e191e6bacbcc214d: 2023-02-11 03:54:24,235 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1676087661058.cb6565dc6679ffd9e191e6bacbcc214d. 2023-02-11 03:54:24,235 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 03:54:24,235 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 03:54:24,235 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:24,235 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-02-11 03:54:24,235 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37011,1676087618428' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-02-11 03:54:24,236 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:24,236 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:24,236 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:24,236 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:54:24,236 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:54:24,237 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:24,237 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 03:54:24,237 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:54:24,237 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:54:24,237 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:24,237 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:24,238 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:54:24,238 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37011,1676087618428' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-02-11 03:54:24,238 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@11a0d8fb[Count = 0] remaining members to acquire global barrier 2023-02-11 03:54:24,238 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-02-11 03:54:24,238 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:24,239 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:24,239 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:24,239 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:24,239 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-02-11 03:54:24,239 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-02-11 03:54:24,239 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,37011,1676087618428' in zk 2023-02-11 03:54:24,239 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:24,239 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 03:54:24,245 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:24,245 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-02-11 03:54:24,245 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:24,245 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:54:24,245 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:54:24,245 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:54:24,245 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-02-11 03:54:24,245 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:54:24,245 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:54:24,245 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:24,246 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:24,246 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:54:24,246 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:24,246 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:24,247 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,37011,1676087618428': 2023-02-11 03:54:24,247 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37011,1676087618428' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-02-11 03:54:24,247 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-02-11 03:54:24,247 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 03:54:24,247 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 03:54:24,247 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:24,247 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 03:54:24,249 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:24,249 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:24,249 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:24,249 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:24,249 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:54:24,249 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:54:24,249 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:24,249 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:24,249 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:54:24,249 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:24,249 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:24,249 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:24,249 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:24,249 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:24,250 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:54:24,250 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:24,250 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:24,250 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:24,251 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:54:24,251 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:24,251 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:24,252 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:24,252 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:24,253 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:24,253 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:24,253 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:24,253 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:54:24,253 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:24,253 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-02-11 03:54:24,253 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 03:54:24,253 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:24,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:24,253 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:24,253 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:24,253 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:24,253 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-02-11 03:54:24,253 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:24,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:24,253 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 03:54:24,353 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 03:54:24,354 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 03:54:24,358 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 03:54:24,360 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-02-11 03:54:24,360 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:24,360 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-02-11 03:54:24,360 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:54:24,360 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-02-11 03:54:24,360 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-02-11 03:54:24,361 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:24,361 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:24,367 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:24,367 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:24,367 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:24,367 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:24,367 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:24,367 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 03:54:24,367 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:24,367 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:24,368 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 03:54:24,368 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:24,368 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:24,368 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-02-11 03:54:24,368 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:24,368 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-02-11 03:54:24,368 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:54:24,368 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-02-11 03:54:24,368 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 03:54:24,368 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 03:54:24,368 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1676087661058.cb6565dc6679ffd9e191e6bacbcc214d. 2023-02-11 03:54:24,368 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1676087661058.cb6565dc6679ffd9e191e6bacbcc214d. started... 2023-02-11 03:54:24,368 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing cb6565dc6679ffd9e191e6bacbcc214d 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-11 03:54:24,781 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512011350f34e0ab6463f9d9c438d2b4a7492 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e201512011350f34e0ab6463f9d9c438d2b4a7492 2023-02-11 03:54:25,187 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyPolicy/cb6565dc6679ffd9e191e6bacbcc214d/.tmp/family1/59eab99167d44ace91e2ae21a7e01ac7 2023-02-11 03:54:25,192 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyPolicy/cb6565dc6679ffd9e191e6bacbcc214d/.tmp/family1/59eab99167d44ace91e2ae21a7e01ac7 as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyPolicy/cb6565dc6679ffd9e191e6bacbcc214d/family1/59eab99167d44ace91e2ae21a7e01ac7 2023-02-11 03:54:25,196 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyPolicy/cb6565dc6679ffd9e191e6bacbcc214d/family1/59eab99167d44ace91e2ae21a7e01ac7, entries=1, sequenceid=13, filesize=4.9 K 2023-02-11 03:54:25,197 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for cb6565dc6679ffd9e191e6bacbcc214d in 829ms, sequenceid=13, compaction requested=true 2023-02-11 03:54:25,197 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for cb6565dc6679ffd9e191e6bacbcc214d: 2023-02-11 03:54:25,197 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1676087661058.cb6565dc6679ffd9e191e6bacbcc214d. 2023-02-11 03:54:25,197 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 03:54:25,197 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 03:54:25,197 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:25,198 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-02-11 03:54:25,198 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37011,1676087618428' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-02-11 03:54:25,199 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:25,199 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:25,199 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:25,199 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:54:25,199 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:54:25,199 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:25,199 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 03:54:25,200 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:54:25,200 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:54:25,200 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:25,200 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:25,200 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:54:25,201 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37011,1676087618428' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-02-11 03:54:25,201 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@397e0d34[Count = 0] remaining members to acquire global barrier 2023-02-11 03:54:25,201 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-02-11 03:54:25,201 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:25,202 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:25,202 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:25,202 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:25,202 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-02-11 03:54:25,202 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-02-11 03:54:25,202 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,37011,1676087618428' in zk 2023-02-11 03:54:25,202 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:25,202 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 03:54:25,203 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-02-11 03:54:25,203 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:25,203 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:54:25,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:25,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:54:25,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:54:25,203 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-02-11 03:54:25,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:54:25,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:54:25,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:25,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:25,205 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:54:25,205 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:25,205 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:25,205 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,37011,1676087618428': 2023-02-11 03:54:25,206 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37011,1676087618428' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-02-11 03:54:25,206 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-02-11 03:54:25,206 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 03:54:25,206 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 03:54:25,206 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:25,206 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 03:54:25,208 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:25,208 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:25,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:25,208 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:25,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:25,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:54:25,208 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:25,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:54:25,209 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:25,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:25,209 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:25,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:54:25,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:25,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:25,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:54:25,210 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:25,210 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:25,210 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:25,210 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:54:25,210 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:25,211 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:25,212 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:25,212 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:25,212 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:25,212 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-02-11 03:54:25,212 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 03:54:25,212 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:54:25,212 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:25,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:25,212 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-02-11 03:54:25,212 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:25,212 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-02-11 03:54:25,212 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:25,212 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:25,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:25,212 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 03:54:25,212 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:25,213 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:25,213 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:25,213 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:25,313 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 03:54:25,314 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 03:54:25,317 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 03:54:25,319 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-02-11 03:54:25,319 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:25,319 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-02-11 03:54:25,319 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:54:25,319 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-02-11 03:54:25,319 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-02-11 03:54:25,320 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:25,320 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:25,321 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:25,321 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:25,321 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:25,321 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:25,321 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:25,321 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 03:54:25,321 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:25,322 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:25,322 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 03:54:25,322 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:25,322 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:25,322 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-02-11 03:54:25,322 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:25,322 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-02-11 03:54:25,322 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:54:25,322 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-02-11 03:54:25,323 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 03:54:25,323 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 03:54:25,323 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1676087661058.cb6565dc6679ffd9e191e6bacbcc214d. 2023-02-11 03:54:25,323 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1676087661058.cb6565dc6679ffd9e191e6bacbcc214d. started... 2023-02-11 03:54:25,323 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing cb6565dc6679ffd9e191e6bacbcc214d 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-11 03:54:25,735 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151205fa62f66f7a8344768337ac83fc6cea05 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151205fa62f66f7a8344768337ac83fc6cea05 2023-02-11 03:54:26,141 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyPolicy/cb6565dc6679ffd9e191e6bacbcc214d/.tmp/family1/f2aa4ab46489463a8e3dd88b56b2ebfc 2023-02-11 03:54:26,146 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyPolicy/cb6565dc6679ffd9e191e6bacbcc214d/.tmp/family1/f2aa4ab46489463a8e3dd88b56b2ebfc as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyPolicy/cb6565dc6679ffd9e191e6bacbcc214d/family1/f2aa4ab46489463a8e3dd88b56b2ebfc 2023-02-11 03:54:26,151 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyPolicy/cb6565dc6679ffd9e191e6bacbcc214d/family1/f2aa4ab46489463a8e3dd88b56b2ebfc, entries=1, sequenceid=17, filesize=4.9 K 2023-02-11 03:54:26,151 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for cb6565dc6679ffd9e191e6bacbcc214d in 828ms, sequenceid=17, compaction requested=true 2023-02-11 03:54:26,152 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for cb6565dc6679ffd9e191e6bacbcc214d: 2023-02-11 03:54:26,152 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1676087661058.cb6565dc6679ffd9e191e6bacbcc214d. 2023-02-11 03:54:26,152 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 03:54:26,152 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 03:54:26,152 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:26,152 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-02-11 03:54:26,152 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37011,1676087618428' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-02-11 03:54:26,153 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:26,153 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:26,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:26,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:54:26,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:54:26,154 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:26,154 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 03:54:26,154 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:54:26,154 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:54:26,154 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:26,154 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:26,155 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:54:26,155 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37011,1676087618428' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-02-11 03:54:26,155 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@82f8645[Count = 0] remaining members to acquire global barrier 2023-02-11 03:54:26,155 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-02-11 03:54:26,155 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:26,157 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:26,157 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:26,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:26,157 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-02-11 03:54:26,157 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-02-11 03:54:26,157 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:26,158 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 03:54:26,158 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,37011,1676087618428' in zk 2023-02-11 03:54:26,159 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:26,159 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-02-11 03:54:26,159 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:26,159 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:54:26,159 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:54:26,159 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:54:26,159 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-02-11 03:54:26,159 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:54:26,159 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:54:26,160 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:26,160 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:26,160 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:54:26,160 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:26,161 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:26,161 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,37011,1676087618428': 2023-02-11 03:54:26,161 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37011,1676087618428' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-02-11 03:54:26,161 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-02-11 03:54:26,161 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 03:54:26,161 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 03:54:26,161 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:26,161 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 03:54:26,163 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:26,163 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:26,163 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:26,163 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:54:26,163 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:54:26,163 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:26,163 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:26,163 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:26,163 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:54:26,163 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:26,163 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:26,163 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:26,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:26,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:26,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:54:26,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:26,168 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:26,168 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:26,168 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:54:26,168 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:26,168 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:26,169 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:26,170 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:26,170 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:26,170 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:26,170 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:26,170 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:54:26,170 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:26,170 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-02-11 03:54:26,170 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:26,170 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-02-11 03:54:26,170 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 03:54:26,170 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:26,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:26,170 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:26,170 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:26,170 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-02-11 03:54:26,170 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 03:54:26,170 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:26,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:26,270 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 03:54:26,271 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 03:54:26,275 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 03:54:26,276 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-02-11 03:54:26,276 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:26,277 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-02-11 03:54:26,277 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:54:26,277 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-02-11 03:54:26,277 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-02-11 03:54:26,277 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:26,277 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:26,279 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:26,279 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:26,279 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:26,279 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:26,279 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:26,279 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 03:54:26,279 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:26,280 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:26,280 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 03:54:26,280 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:26,280 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:26,280 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-02-11 03:54:26,280 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:26,280 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-02-11 03:54:26,280 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:54:26,280 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-02-11 03:54:26,281 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 03:54:26,281 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 03:54:26,281 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1676087661058.cb6565dc6679ffd9e191e6bacbcc214d. 2023-02-11 03:54:26,281 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1676087661058.cb6565dc6679ffd9e191e6bacbcc214d. started... 2023-02-11 03:54:26,281 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing cb6565dc6679ffd9e191e6bacbcc214d 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-11 03:54:26,693 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151228323c92981aa34047a462aa33a1605726 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151228323c92981aa34047a462aa33a1605726 2023-02-11 03:54:27,098 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=21, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyPolicy/cb6565dc6679ffd9e191e6bacbcc214d/.tmp/family1/7a1d23c62ba24e2bbc522363c68590b6 2023-02-11 03:54:27,104 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyPolicy/cb6565dc6679ffd9e191e6bacbcc214d/.tmp/family1/7a1d23c62ba24e2bbc522363c68590b6 as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyPolicy/cb6565dc6679ffd9e191e6bacbcc214d/family1/7a1d23c62ba24e2bbc522363c68590b6 2023-02-11 03:54:27,109 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyPolicy/cb6565dc6679ffd9e191e6bacbcc214d/family1/7a1d23c62ba24e2bbc522363c68590b6, entries=1, sequenceid=21, filesize=4.9 K 2023-02-11 03:54:27,110 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for cb6565dc6679ffd9e191e6bacbcc214d in 829ms, sequenceid=21, compaction requested=true 2023-02-11 03:54:27,110 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for cb6565dc6679ffd9e191e6bacbcc214d: 2023-02-11 03:54:27,110 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1676087661058.cb6565dc6679ffd9e191e6bacbcc214d. 2023-02-11 03:54:27,110 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 03:54:27,110 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 03:54:27,110 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:27,110 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-02-11 03:54:27,110 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37011,1676087618428' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-02-11 03:54:27,113 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:27,113 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:27,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:27,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:54:27,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:54:27,114 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:27,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:54:27,114 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 03:54:27,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:54:27,115 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:27,115 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:27,115 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:54:27,115 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37011,1676087618428' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-02-11 03:54:27,116 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@75fac7ee[Count = 0] remaining members to acquire global barrier 2023-02-11 03:54:27,116 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-02-11 03:54:27,116 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:27,117 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:27,117 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:27,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:27,117 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-02-11 03:54:27,117 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-02-11 03:54:27,117 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,37011,1676087618428' in zk 2023-02-11 03:54:27,117 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:27,117 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 03:54:27,118 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:27,118 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-02-11 03:54:27,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:27,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:54:27,119 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:54:27,118 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:54:27,119 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-02-11 03:54:27,119 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:54:27,119 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:54:27,120 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:27,120 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:27,121 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:54:27,121 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:27,121 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:27,122 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,37011,1676087618428': 2023-02-11 03:54:27,122 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37011,1676087618428' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-02-11 03:54:27,122 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-02-11 03:54:27,122 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 03:54:27,122 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 03:54:27,122 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:27,122 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 03:54:27,125 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:27,125 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:27,125 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:27,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:27,125 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:27,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:27,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:54:27,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:54:27,125 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:27,125 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:27,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:27,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:54:27,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:27,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:27,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:54:27,126 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:27,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:27,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:27,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:54:27,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:27,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:27,128 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:27,128 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:27,128 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:27,128 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:27,128 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:27,128 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:27,128 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:27,128 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:27,129 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:27,129 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:27,129 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-02-11 03:54:27,129 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:27,129 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 03:54:27,129 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-02-11 03:54:27,129 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:54:27,129 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-02-11 03:54:27,129 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 03:54:27,129 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:27,129 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:27,229 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 03:54:27,230 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 03:54:27,235 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 03:54:27,237 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-02-11 03:54:27,237 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:27,237 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-02-11 03:54:27,237 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:54:27,238 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-02-11 03:54:27,238 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-02-11 03:54:27,238 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:27,238 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:27,239 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:27,239 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:27,239 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:27,239 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:27,240 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:27,240 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 03:54:27,240 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:27,240 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:27,240 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 03:54:27,240 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:27,240 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:27,241 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-02-11 03:54:27,241 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:27,241 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-02-11 03:54:27,241 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:54:27,241 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-02-11 03:54:27,241 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 03:54:27,241 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 03:54:27,241 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1676087661058.cb6565dc6679ffd9e191e6bacbcc214d. 2023-02-11 03:54:27,241 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1676087661058.cb6565dc6679ffd9e191e6bacbcc214d. started... 2023-02-11 03:54:27,241 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing cb6565dc6679ffd9e191e6bacbcc214d 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-11 03:54:27,256 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151231c21f41a3804945988d9907edaa6bca0f to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151231c21f41a3804945988d9907edaa6bca0f 2023-02-11 03:54:27,663 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=25, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyPolicy/cb6565dc6679ffd9e191e6bacbcc214d/.tmp/family1/1beb6a91f71047f1804d38be4477d783 2023-02-11 03:54:27,668 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyPolicy/cb6565dc6679ffd9e191e6bacbcc214d/.tmp/family1/1beb6a91f71047f1804d38be4477d783 as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyPolicy/cb6565dc6679ffd9e191e6bacbcc214d/family1/1beb6a91f71047f1804d38be4477d783 2023-02-11 03:54:27,672 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyPolicy/cb6565dc6679ffd9e191e6bacbcc214d/family1/1beb6a91f71047f1804d38be4477d783, entries=1, sequenceid=25, filesize=4.9 K 2023-02-11 03:54:27,673 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for cb6565dc6679ffd9e191e6bacbcc214d in 432ms, sequenceid=25, compaction requested=true 2023-02-11 03:54:27,673 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for cb6565dc6679ffd9e191e6bacbcc214d: 2023-02-11 03:54:27,673 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1676087661058.cb6565dc6679ffd9e191e6bacbcc214d. 2023-02-11 03:54:27,673 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 03:54:27,673 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 03:54:27,673 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:27,673 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-02-11 03:54:27,673 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37011,1676087618428' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-02-11 03:54:27,674 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:27,674 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:27,675 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:27,675 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:54:27,675 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:54:27,675 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:27,675 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 03:54:27,675 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:54:27,675 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:54:27,675 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:27,676 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:27,676 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:54:27,676 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37011,1676087618428' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-02-11 03:54:27,676 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@628aaeaa[Count = 0] remaining members to acquire global barrier 2023-02-11 03:54:27,676 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-02-11 03:54:27,676 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:27,678 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:27,678 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:27,678 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:27,678 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-02-11 03:54:27,678 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-02-11 03:54:27,678 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,37011,1676087618428' in zk 2023-02-11 03:54:27,678 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:27,679 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 03:54:27,680 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-02-11 03:54:27,680 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:27,680 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:54:27,680 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:27,680 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:54:27,680 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:54:27,680 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-02-11 03:54:27,680 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:54:27,680 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:54:27,681 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:27,681 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:27,681 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:54:27,681 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:27,681 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:27,682 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,37011,1676087618428': 2023-02-11 03:54:27,682 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37011,1676087618428' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-02-11 03:54:27,682 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-02-11 03:54:27,682 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 03:54:27,682 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 03:54:27,682 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:27,682 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 03:54:27,683 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:27,683 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:27,683 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:27,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:27,684 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:54:27,684 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:54:27,683 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:27,684 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:27,684 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:54:27,684 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:27,684 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:27,684 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:27,684 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:27,684 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:27,685 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:54:27,685 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:27,685 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:27,685 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:27,685 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:54:27,686 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:27,686 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:27,687 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:27,687 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:27,687 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:27,687 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-02-11 03:54:27,687 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-02-11 03:54:27,687 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 03:54:27,687 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:27,687 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:54:27,687 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:27,687 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:27,687 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:27,688 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-02-11 03:54:27,688 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:27,688 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 03:54:27,688 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:27,688 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:27,688 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:27,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:27,688 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:27,788 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 03:54:27,789 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 03:54:27,792 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 03:54:27,794 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-02-11 03:54:27,794 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:27,794 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-02-11 03:54:27,794 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:54:27,794 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-02-11 03:54:27,794 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-02-11 03:54:27,794 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:27,794 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:27,797 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:27,797 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:27,797 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:27,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:27,797 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:27,797 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 03:54:27,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:27,798 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:27,798 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 03:54:27,798 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:27,798 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:27,798 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-02-11 03:54:27,798 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:27,798 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-02-11 03:54:27,798 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:54:27,798 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-02-11 03:54:27,799 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 03:54:27,799 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 03:54:27,799 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1676087661058.cb6565dc6679ffd9e191e6bacbcc214d. 2023-02-11 03:54:27,799 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1676087661058.cb6565dc6679ffd9e191e6bacbcc214d. started... 2023-02-11 03:54:27,799 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing cb6565dc6679ffd9e191e6bacbcc214d 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-11 03:54:28,212 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160101c6c524c9bb104b9a9bd465dd2d517c07 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20160101c6c524c9bb104b9a9bd465dd2d517c07 2023-02-11 03:54:28,617 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=29, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyPolicy/cb6565dc6679ffd9e191e6bacbcc214d/.tmp/family1/49dd9a08fe9b433fbf1a6eec3893827e 2023-02-11 03:54:28,622 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyPolicy/cb6565dc6679ffd9e191e6bacbcc214d/.tmp/family1/49dd9a08fe9b433fbf1a6eec3893827e as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyPolicy/cb6565dc6679ffd9e191e6bacbcc214d/family1/49dd9a08fe9b433fbf1a6eec3893827e 2023-02-11 03:54:28,626 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyPolicy/cb6565dc6679ffd9e191e6bacbcc214d/family1/49dd9a08fe9b433fbf1a6eec3893827e, entries=1, sequenceid=29, filesize=4.9 K 2023-02-11 03:54:28,627 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for cb6565dc6679ffd9e191e6bacbcc214d in 828ms, sequenceid=29, compaction requested=true 2023-02-11 03:54:28,627 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for cb6565dc6679ffd9e191e6bacbcc214d: 2023-02-11 03:54:28,627 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1676087661058.cb6565dc6679ffd9e191e6bacbcc214d. 2023-02-11 03:54:28,627 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 03:54:28,627 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 03:54:28,627 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:28,627 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-02-11 03:54:28,627 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37011,1676087618428' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-02-11 03:54:28,628 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:28,628 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:28,628 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:28,629 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:54:28,629 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:54:28,629 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:28,629 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 03:54:28,629 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:54:28,629 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:54:28,629 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:28,629 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:28,630 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:54:28,630 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37011,1676087618428' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-02-11 03:54:28,630 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2b541862[Count = 0] remaining members to acquire global barrier 2023-02-11 03:54:28,630 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-02-11 03:54:28,630 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:28,631 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:28,631 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:28,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:28,631 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-02-11 03:54:28,631 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-02-11 03:54:28,631 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:28,631 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 03:54:28,631 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,37011,1676087618428' in zk 2023-02-11 03:54:28,632 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-02-11 03:54:28,632 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:54:28,632 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:28,632 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-02-11 03:54:28,632 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:28,632 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:54:28,632 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:54:28,633 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:54:28,633 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:54:28,633 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:28,633 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:28,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:54:28,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:28,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:28,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,37011,1676087618428': 2023-02-11 03:54:28,634 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37011,1676087618428' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-02-11 03:54:28,634 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-02-11 03:54:28,635 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 03:54:28,635 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 03:54:28,635 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:28,635 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 03:54:28,637 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:28,637 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:28,637 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:28,637 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:28,637 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:28,637 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:54:28,637 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:54:28,637 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:28,637 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:28,637 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:54:28,637 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:28,638 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:28,638 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:28,638 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:28,638 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:54:28,638 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:28,638 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:28,638 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:28,639 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:54:28,639 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:28,639 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:28,640 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:28,640 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:28,640 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:28,640 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:54:28,640 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:28,640 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:28,641 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:28,640 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-02-11 03:54:28,641 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 03:54:28,640 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:28,641 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:28,641 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:28,641 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:28,641 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:28,641 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:28,641 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:28,641 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-02-11 03:54:28,641 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 03:54:28,741 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 03:54:28,742 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 03:54:28,746 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 03:54:28,748 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-02-11 03:54:28,748 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:28,748 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-02-11 03:54:28,748 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:54:28,748 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-02-11 03:54:28,748 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-02-11 03:54:28,748 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:28,748 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:28,754 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:28,754 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:28,754 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:28,754 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:28,754 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:28,754 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 03:54:28,754 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:28,754 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:28,755 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 03:54:28,755 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:28,755 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:28,755 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-02-11 03:54:28,755 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:28,755 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-02-11 03:54:28,755 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:54:28,755 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-02-11 03:54:28,755 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 03:54:28,755 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 03:54:28,755 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1676087661058.cb6565dc6679ffd9e191e6bacbcc214d. 2023-02-11 03:54:28,755 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1676087661058.cb6565dc6679ffd9e191e6bacbcc214d. started... 2023-02-11 03:54:28,755 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing cb6565dc6679ffd9e191e6bacbcc214d 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-11 03:54:29,167 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160103165ff5d456ec45b0907ce0d70167e96a to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20160103165ff5d456ec45b0907ce0d70167e96a 2023-02-11 03:54:29,573 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=33, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyPolicy/cb6565dc6679ffd9e191e6bacbcc214d/.tmp/family1/6820fa5bf43945d39ae117a760887672 2023-02-11 03:54:29,578 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyPolicy/cb6565dc6679ffd9e191e6bacbcc214d/.tmp/family1/6820fa5bf43945d39ae117a760887672 as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyPolicy/cb6565dc6679ffd9e191e6bacbcc214d/family1/6820fa5bf43945d39ae117a760887672 2023-02-11 03:54:29,583 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyPolicy/cb6565dc6679ffd9e191e6bacbcc214d/family1/6820fa5bf43945d39ae117a760887672, entries=1, sequenceid=33, filesize=4.9 K 2023-02-11 03:54:29,583 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for cb6565dc6679ffd9e191e6bacbcc214d in 828ms, sequenceid=33, compaction requested=true 2023-02-11 03:54:29,583 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for cb6565dc6679ffd9e191e6bacbcc214d: 2023-02-11 03:54:29,584 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1676087661058.cb6565dc6679ffd9e191e6bacbcc214d. 2023-02-11 03:54:29,584 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 03:54:29,584 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 03:54:29,584 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:29,584 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-02-11 03:54:29,584 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37011,1676087618428' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-02-11 03:54:29,585 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:29,585 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:29,585 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:29,585 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:54:29,586 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:54:29,586 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:29,586 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 03:54:29,586 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:54:29,586 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:54:29,586 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:29,586 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:29,587 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:54:29,587 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37011,1676087618428' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-02-11 03:54:29,587 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5f3dbee3[Count = 0] remaining members to acquire global barrier 2023-02-11 03:54:29,587 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-02-11 03:54:29,587 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:29,588 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:29,588 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:29,588 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:29,588 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-02-11 03:54:29,588 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-02-11 03:54:29,588 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,37011,1676087618428' in zk 2023-02-11 03:54:29,588 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:29,588 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 03:54:29,589 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:29,589 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:29,589 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-02-11 03:54:29,590 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:54:29,589 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:54:29,590 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:54:29,590 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-02-11 03:54:29,590 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:54:29,590 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:54:29,590 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:29,591 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:29,591 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:54:29,591 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:29,591 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:29,592 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,37011,1676087618428': 2023-02-11 03:54:29,592 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37011,1676087618428' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-02-11 03:54:29,592 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-02-11 03:54:29,592 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 03:54:29,592 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 03:54:29,592 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:29,592 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 03:54:29,594 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:29,594 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:29,594 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:29,594 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:54:29,594 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:54:29,594 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:29,594 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:29,594 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:29,594 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:54:29,594 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:29,594 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:29,595 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:29,595 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:29,595 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:29,595 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:54:29,595 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:29,595 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:29,595 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:29,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:54:29,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:29,597 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:29,598 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:29,598 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:29,598 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:29,598 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:29,598 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-02-11 03:54:29,598 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 03:54:29,598 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:29,598 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:54:29,598 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:29,599 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:29,599 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:29,599 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-02-11 03:54:29,599 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:29,599 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:29,599 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:29,599 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:29,599 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:29,600 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-02-11 03:54:29,600 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 03:54:29,700 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 03:54:29,701 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 03:54:29,705 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 03:54:29,706 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-02-11 03:54:29,706 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:29,706 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-02-11 03:54:29,706 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:54:29,707 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-02-11 03:54:29,707 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-02-11 03:54:29,707 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:29,707 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:29,708 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:29,708 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:29,708 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:29,708 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:29,708 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:29,708 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 03:54:29,709 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:29,709 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:29,709 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 03:54:29,709 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:29,709 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:29,709 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-02-11 03:54:29,709 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:29,709 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-02-11 03:54:29,709 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:54:29,709 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-02-11 03:54:29,710 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 03:54:29,710 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 03:54:29,710 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1676087661058.cb6565dc6679ffd9e191e6bacbcc214d. 2023-02-11 03:54:29,710 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1676087661058.cb6565dc6679ffd9e191e6bacbcc214d. started... 2023-02-11 03:54:29,710 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing cb6565dc6679ffd9e191e6bacbcc214d 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-11 03:54:30,123 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015090781f32b47a21f4b6bad644a841c1852a6 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e2015090781f32b47a21f4b6bad644a841c1852a6 2023-02-11 03:54:30,130 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=37, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyPolicy/cb6565dc6679ffd9e191e6bacbcc214d/.tmp/family1/396ebaa606d9430e93f9a483a4aa749b 2023-02-11 03:54:30,135 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyPolicy/cb6565dc6679ffd9e191e6bacbcc214d/.tmp/family1/396ebaa606d9430e93f9a483a4aa749b as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyPolicy/cb6565dc6679ffd9e191e6bacbcc214d/family1/396ebaa606d9430e93f9a483a4aa749b 2023-02-11 03:54:30,140 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyPolicy/cb6565dc6679ffd9e191e6bacbcc214d/family1/396ebaa606d9430e93f9a483a4aa749b, entries=1, sequenceid=37, filesize=4.9 K 2023-02-11 03:54:30,141 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for cb6565dc6679ffd9e191e6bacbcc214d in 431ms, sequenceid=37, compaction requested=true 2023-02-11 03:54:30,141 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for cb6565dc6679ffd9e191e6bacbcc214d: 2023-02-11 03:54:30,141 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1676087661058.cb6565dc6679ffd9e191e6bacbcc214d. 2023-02-11 03:54:30,141 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 03:54:30,141 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 03:54:30,141 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:30,141 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-02-11 03:54:30,141 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37011,1676087618428' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-02-11 03:54:30,143 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:30,143 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:30,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:30,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:54:30,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:54:30,143 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:30,143 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 03:54:30,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:54:30,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:54:30,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:30,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:30,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:54:30,145 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37011,1676087618428' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-02-11 03:54:30,145 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-02-11 03:54:30,145 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7c5bd75d[Count = 0] remaining members to acquire global barrier 2023-02-11 03:54:30,145 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:30,147 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:30,147 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:30,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:30,147 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-02-11 03:54:30,147 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-02-11 03:54:30,147 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,37011,1676087618428' in zk 2023-02-11 03:54:30,147 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:30,147 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 03:54:30,148 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:30,148 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-02-11 03:54:30,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:30,149 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:54:30,149 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:54:30,149 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:54:30,149 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-02-11 03:54:30,149 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:54:30,149 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:54:30,150 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:30,150 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:30,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:54:30,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:30,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:30,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,37011,1676087618428': 2023-02-11 03:54:30,152 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37011,1676087618428' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-02-11 03:54:30,152 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-02-11 03:54:30,152 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 03:54:30,152 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 03:54:30,152 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:30,152 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 03:54:30,154 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:30,154 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:30,154 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:30,154 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:54:30,154 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:54:30,154 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:30,154 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:30,154 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:30,154 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:54:30,154 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:30,154 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:30,154 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:30,154 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:30,155 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:30,155 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:54:30,155 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:30,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:30,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:54:30,156 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:30,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:30,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:30,158 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:30,158 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:30,158 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:30,158 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:30,158 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:54:30,158 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:30,158 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:30,158 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:30,158 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-02-11 03:54:30,158 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 03:54:30,158 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:30,158 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:30,159 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:30,159 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:30,159 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-02-11 03:54:30,159 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:30,159 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:30,159 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 03:54:30,259 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 03:54:30,260 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 03:54:30,264 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 03:54:30,266 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-02-11 03:54:30,266 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:30,266 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-02-11 03:54:30,266 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:54:30,266 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-02-11 03:54:30,266 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-02-11 03:54:30,266 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:30,266 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:30,268 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:30,268 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:30,268 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:30,268 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:30,269 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:30,269 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 03:54:30,269 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:30,269 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:30,269 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 03:54:30,269 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:30,269 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:30,270 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-02-11 03:54:30,270 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:30,270 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-02-11 03:54:30,270 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:54:30,270 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-02-11 03:54:30,270 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 03:54:30,270 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 03:54:30,270 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1676087661058.cb6565dc6679ffd9e191e6bacbcc214d. 2023-02-11 03:54:30,270 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1676087661058.cb6565dc6679ffd9e191e6bacbcc214d. started... 2023-02-11 03:54:30,270 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing cb6565dc6679ffd9e191e6bacbcc214d 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-11 03:54:30,683 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151120776efc3cf49c4761aa7fde56b50154a4 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151120776efc3cf49c4761aa7fde56b50154a4 2023-02-11 03:54:31,089 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=41, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyPolicy/cb6565dc6679ffd9e191e6bacbcc214d/.tmp/family1/e19ff7eef6e74aec94c68ccd911bea1b 2023-02-11 03:54:31,095 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyPolicy/cb6565dc6679ffd9e191e6bacbcc214d/.tmp/family1/e19ff7eef6e74aec94c68ccd911bea1b as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyPolicy/cb6565dc6679ffd9e191e6bacbcc214d/family1/e19ff7eef6e74aec94c68ccd911bea1b 2023-02-11 03:54:31,100 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyPolicy/cb6565dc6679ffd9e191e6bacbcc214d/family1/e19ff7eef6e74aec94c68ccd911bea1b, entries=1, sequenceid=41, filesize=4.9 K 2023-02-11 03:54:31,100 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for cb6565dc6679ffd9e191e6bacbcc214d in 830ms, sequenceid=41, compaction requested=true 2023-02-11 03:54:31,101 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for cb6565dc6679ffd9e191e6bacbcc214d: 2023-02-11 03:54:31,101 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1676087661058.cb6565dc6679ffd9e191e6bacbcc214d. 2023-02-11 03:54:31,101 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 03:54:31,101 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 03:54:31,101 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:31,101 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-02-11 03:54:31,101 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37011,1676087618428' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-02-11 03:54:31,102 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:31,102 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:31,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:31,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:54:31,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:54:31,103 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:31,103 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 03:54:31,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:54:31,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:54:31,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:31,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:31,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:54:31,104 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37011,1676087618428' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-02-11 03:54:31,104 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4b93d982[Count = 0] remaining members to acquire global barrier 2023-02-11 03:54:31,104 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-02-11 03:54:31,104 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:31,105 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:31,105 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:31,105 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:31,105 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-02-11 03:54:31,105 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-02-11 03:54:31,105 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,37011,1676087618428' in zk 2023-02-11 03:54:31,105 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:31,105 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 03:54:31,106 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:31,106 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-02-11 03:54:31,106 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:31,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:54:31,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:54:31,107 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:54:31,107 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-02-11 03:54:31,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:54:31,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:54:31,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:31,108 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:31,108 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:54:31,108 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:31,109 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:31,109 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,37011,1676087618428': 2023-02-11 03:54:31,109 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37011,1676087618428' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-02-11 03:54:31,109 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-02-11 03:54:31,109 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 03:54:31,109 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 03:54:31,109 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:31,109 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 03:54:31,113 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:31,113 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:31,113 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:31,113 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:54:31,113 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:54:31,113 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:31,113 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:31,113 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:31,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:54:31,114 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:31,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:31,114 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:31,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:31,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:31,115 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:54:31,115 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:31,115 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:31,115 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:31,115 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:54:31,116 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:31,116 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:31,120 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:31,120 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:31,120 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:31,121 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-02-11 03:54:31,121 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:31,121 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 03:54:31,121 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:31,120 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:31,120 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:31,120 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:54:31,121 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:31,121 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:31,121 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-02-11 03:54:31,121 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:31,121 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 03:54:31,121 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:31,121 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:31,121 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:31,221 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 03:54:31,222 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 03:54:31,223 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=39921] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}] of the table testMajorCompactionWithWeeklyPolicy 2023-02-11 03:54:31,225 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676087621935] procedure2.ProcedureExecutor(1029): Stored pid=38, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE 2023-02-11 03:54:31,226 DEBUG [PEWorker-4] locking.LockProcedure(309): LOCKED pid=38, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE 2023-02-11 03:54:31,227 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676087621935] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyPolicy', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-11 03:54:31,227 INFO [PEWorker-4] procedure2.TimeoutExecutorThread(81): ADDED pid=38, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1676088271227 2023-02-11 03:54:31,228 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676087621935] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-02-11 03:54:31,230 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676087621935] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 10 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyPolicy' and column 'family1' 2023-02-11 03:54:31,230 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676087621935] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyPolicy' column='family1' 2023-02-11 03:54:31,230 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676087621935] zookeeper.ReadOnlyZKClient(139): Connect 0x70415025 to 127.0.0.1:53383 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-11 03:54:31,236 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676087621935] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@68f7a1df, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-11 03:54:31,236 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-02-11 03:54:31,686 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201601030a9488a6fab9408095302bf50e9e0378 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e201601030a9488a6fab9408095302bf50e9e0378 2023-02-11 03:54:31,687 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMajorCompactionWithWeeklyPolicy' column='family1' 2023-02-11 03:54:31,699 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-11 03:54:31,701 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:49750, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-11 03:54:31,706 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-02-11 03:54:31,707 INFO [RS-EventLoopGroup-1-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:46356, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-02-11 03:54:31,715 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228/family1/4d73eb5206f346098fcde0d59cc82434 first=Optional[r3] last=Optional[r6] 2023-02-11 03:54:31,715 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionWithWeeklyPolicy,,1676087661058.cb6565dc6679ffd9e191e6bacbcc214d., hostname=jenkins-hbase4.apache.org,37011,1676087618428, seqNum=2 for row with hfile group [{family1,hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228/family1/4d73eb5206f346098fcde0d59cc82434}] 2023-02-11 03:54:31,720 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.HStore(763): Validating hfile at hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228/family1/4d73eb5206f346098fcde0d59cc82434 for inclusion in cb6565dc6679ffd9e191e6bacbcc214d/family1 2023-02-11 03:54:31,722 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.HStore(775): HFile bounds: first=r3 last=r6 2023-02-11 03:54:31,722 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.HStore(777): Region bounds: first= last= 2023-02-11 03:54:31,722 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.HRegion(2445): Flush status journal for cb6565dc6679ffd9e191e6bacbcc214d: 2023-02-11 03:54:31,723 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228/family1/4d73eb5206f346098fcde0d59cc82434 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/staging/jenkins__testMajorCompactionWithWeeklyPolicy__fhs1e8gao80d9e9ojvhv911ia63nbbo72v6jioneholceuig0g61l57ocq9mi4f/family1/4d73eb5206f346098fcde0d59cc82434 2023-02-11 03:54:31,724 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/staging/jenkins__testMajorCompactionWithWeeklyPolicy__fhs1e8gao80d9e9ojvhv911ia63nbbo72v6jioneholceuig0g61l57ocq9mi4f/family1/4d73eb5206f346098fcde0d59cc82434 as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyPolicy/cb6565dc6679ffd9e191e6bacbcc214d/family1/4a3ecd528f104618966a5aafcca91ce0_SeqId_44_ 2023-02-11 03:54:31,725 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.HStore(849): Loaded HFile hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/staging/jenkins__testMajorCompactionWithWeeklyPolicy__fhs1e8gao80d9e9ojvhv911ia63nbbo72v6jioneholceuig0g61l57ocq9mi4f/family1/4d73eb5206f346098fcde0d59cc82434 into cb6565dc6679ffd9e191e6bacbcc214d/family1 as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyPolicy/cb6565dc6679ffd9e191e6bacbcc214d/family1/4a3ecd528f104618966a5aafcca91ce0_SeqId_44_ - updating store file list. 2023-02-11 03:54:31,728 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.HStore(882): Loaded HFile hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyPolicy/cb6565dc6679ffd9e191e6bacbcc214d/family1/4a3ecd528f104618966a5aafcca91ce0_SeqId_44_ into cb6565dc6679ffd9e191e6bacbcc214d/family1 2023-02-11 03:54:31,728 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.HStore(855): Successfully loaded hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/staging/jenkins__testMajorCompactionWithWeeklyPolicy__fhs1e8gao80d9e9ojvhv911ia63nbbo72v6jioneholceuig0g61l57ocq9mi4f/family1/4d73eb5206f346098fcde0d59cc82434 into cb6565dc6679ffd9e191e6bacbcc214d/family1 (new location: hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyPolicy/cb6565dc6679ffd9e191e6bacbcc214d/family1/4a3ecd528f104618966a5aafcca91ce0_SeqId_44_) 2023-02-11 03:54:31,729 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/staging/jenkins__testMajorCompactionWithWeeklyPolicy__fhs1e8gao80d9e9ojvhv911ia63nbbo72v6jioneholceuig0g61l57ocq9mi4f/family1/4d73eb5206f346098fcde0d59cc82434 2023-02-11 03:54:31,729 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] compactions.SortedCompactionPolicy(75): Selecting compaction from 11 store files, 0 compacting, 11 eligible, 16 blocking 2023-02-11 03:54:31,732 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 10 files of size 51214 starting at candidate #1 after considering 44 permutations with 44 in ratio 2023-02-11 03:54:31,732 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.HStore(1912): cb6565dc6679ffd9e191e6bacbcc214d/family1 is initiating minor compaction 2023-02-11 03:54:31,732 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@716f71e5; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-11 03:54:31,732 INFO [RS:0;jenkins-hbase4:37011-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of cb6565dc6679ffd9e191e6bacbcc214d/family1 in testMajorCompactionWithWeeklyPolicy,,1676087661058.cb6565dc6679ffd9e191e6bacbcc214d. 2023-02-11 03:54:31,732 INFO [RS:0;jenkins-hbase4:37011-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyPolicy/cb6565dc6679ffd9e191e6bacbcc214d/family1/365ad5fc2a874c7eac0ce82e29ac3375, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyPolicy/cb6565dc6679ffd9e191e6bacbcc214d/family1/59eab99167d44ace91e2ae21a7e01ac7, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyPolicy/cb6565dc6679ffd9e191e6bacbcc214d/family1/f2aa4ab46489463a8e3dd88b56b2ebfc, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyPolicy/cb6565dc6679ffd9e191e6bacbcc214d/family1/7a1d23c62ba24e2bbc522363c68590b6, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyPolicy/cb6565dc6679ffd9e191e6bacbcc214d/family1/1beb6a91f71047f1804d38be4477d783, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyPolicy/cb6565dc6679ffd9e191e6bacbcc214d/family1/49dd9a08fe9b433fbf1a6eec3893827e, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyPolicy/cb6565dc6679ffd9e191e6bacbcc214d/family1/6820fa5bf43945d39ae117a760887672, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyPolicy/cb6565dc6679ffd9e191e6bacbcc214d/family1/396ebaa606d9430e93f9a483a4aa749b, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyPolicy/cb6565dc6679ffd9e191e6bacbcc214d/family1/e19ff7eef6e74aec94c68ccd911bea1b, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyPolicy/cb6565dc6679ffd9e191e6bacbcc214d/family1/4a3ecd528f104618966a5aafcca91ce0_SeqId_44_] into tmpdir=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyPolicy/cb6565dc6679ffd9e191e6bacbcc214d/.tmp, totalSize=50.0 K 2023-02-11 03:54:31,732 DEBUG [RS:0;jenkins-hbase4:37011-shortCompactions-0] compactions.Compactor(207): Compacting 365ad5fc2a874c7eac0ce82e29ac3375, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=9 2023-02-11 03:54:31,732 DEBUG [RS:0;jenkins-hbase4:37011-shortCompactions-0] compactions.Compactor(207): Compacting 59eab99167d44ace91e2ae21a7e01ac7, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=13 2023-02-11 03:54:31,732 DEBUG [RS:0;jenkins-hbase4:37011-shortCompactions-0] compactions.Compactor(207): Compacting f2aa4ab46489463a8e3dd88b56b2ebfc, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=17 2023-02-11 03:54:31,732 DEBUG [RS:0;jenkins-hbase4:37011-shortCompactions-0] compactions.Compactor(207): Compacting 7a1d23c62ba24e2bbc522363c68590b6, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=21 2023-02-11 03:54:31,732 DEBUG [RS:0;jenkins-hbase4:37011-shortCompactions-0] compactions.Compactor(207): Compacting 1beb6a91f71047f1804d38be4477d783, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=25 2023-02-11 03:54:31,733 DEBUG [RS:0;jenkins-hbase4:37011-shortCompactions-0] compactions.Compactor(207): Compacting 49dd9a08fe9b433fbf1a6eec3893827e, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=29 2023-02-11 03:54:31,733 DEBUG [RS:0;jenkins-hbase4:37011-shortCompactions-0] compactions.Compactor(207): Compacting 6820fa5bf43945d39ae117a760887672, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=33 2023-02-11 03:54:31,733 DEBUG [RS:0;jenkins-hbase4:37011-shortCompactions-0] compactions.Compactor(207): Compacting 396ebaa606d9430e93f9a483a4aa749b, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=37 2023-02-11 03:54:31,733 DEBUG [RS:0;jenkins-hbase4:37011-shortCompactions-0] compactions.Compactor(207): Compacting e19ff7eef6e74aec94c68ccd911bea1b, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=41 2023-02-11 03:54:31,733 DEBUG [RS:0;jenkins-hbase4:37011-shortCompactions-0] compactions.Compactor(207): Compacting 4a3ecd528f104618966a5aafcca91ce0_SeqId_44_, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=45 2023-02-11 03:54:31,734 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMajorCompactionWithWeeklyPolicy' column='family1' 2023-02-11 03:54:31,735 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-11 03:54:31,736 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151228323c92981aa34047a462aa33a1605726 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151228323c92981aa34047a462aa33a1605726 2023-02-11 03:54:31,738 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151231c21f41a3804945988d9907edaa6bca0f to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151231c21f41a3804945988d9907edaa6bca0f 2023-02-11 03:54:31,740 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20160101c6c524c9bb104b9a9bd465dd2d517c07 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20160101c6c524c9bb104b9a9bd465dd2d517c07 2023-02-11 03:54:31,741 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20160103165ff5d456ec45b0907ce0d70167e96a to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20160103165ff5d456ec45b0907ce0d70167e96a 2023-02-11 03:54:31,741 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyPolicy 4 expired mob files are deleted 2023-02-11 03:54:31,741 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228' 2023-02-11 03:54:31,742 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-02-11 03:54:31,742 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-02-11 03:54:31,742 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-02-11 03:54:31,742 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-02-11 03:54:31,742 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-02-11 03:54:31,742 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-02-11 03:54:31,742 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-02-11 03:54:31,742 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-02-11 03:54:31,768 INFO [RS:0;jenkins-hbase4:37011-shortCompactions-0] throttle.PressureAwareThroughputController(145): cb6565dc6679ffd9e191e6bacbcc214d#family1#compaction#49 average throughput is 0.68 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-11 03:54:31,810 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015120561559b55917f4145a085e7840e2ba777 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e2015120561559b55917f4145a085e7840e2ba777 2023-02-11 03:54:31,811 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMajorCompactionWithWeeklyPolicy' column='family1' 2023-02-11 03:54:31,836 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130/family1/28fea89f043e430b8c307e0a2d124d72 first=Optional[r00] last=Optional[r2] 2023-02-11 03:54:31,837 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionWithWeeklyPolicy,,1676087661058.cb6565dc6679ffd9e191e6bacbcc214d., hostname=jenkins-hbase4.apache.org,37011,1676087618428, seqNum=2 for row with hfile group [{family1,hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130/family1/28fea89f043e430b8c307e0a2d124d72}] 2023-02-11 03:54:31,850 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.HStore(763): Validating hfile at hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130/family1/28fea89f043e430b8c307e0a2d124d72 for inclusion in cb6565dc6679ffd9e191e6bacbcc214d/family1 2023-02-11 03:54:31,854 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.HStore(775): HFile bounds: first=r00 last=r2 2023-02-11 03:54:31,854 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.HStore(777): Region bounds: first= last= 2023-02-11 03:54:31,854 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.HRegion(2445): Flush status journal for cb6565dc6679ffd9e191e6bacbcc214d: 2023-02-11 03:54:31,855 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130/family1/28fea89f043e430b8c307e0a2d124d72 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/staging/jenkins__testMajorCompactionWithWeeklyPolicy__8e5ngphak3rhabrhku4umtqriidrofls34u99r7r1k5v3e3ke7jen4ecohstpa72/family1/28fea89f043e430b8c307e0a2d124d72 2023-02-11 03:54:31,858 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/staging/jenkins__testMajorCompactionWithWeeklyPolicy__8e5ngphak3rhabrhku4umtqriidrofls34u99r7r1k5v3e3ke7jen4ecohstpa72/family1/28fea89f043e430b8c307e0a2d124d72 as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyPolicy/cb6565dc6679ffd9e191e6bacbcc214d/family1/24ffe0948a27475190dd3f812406b170_SeqId_46_ 2023-02-11 03:54:31,858 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.HStore(849): Loaded HFile hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/staging/jenkins__testMajorCompactionWithWeeklyPolicy__8e5ngphak3rhabrhku4umtqriidrofls34u99r7r1k5v3e3ke7jen4ecohstpa72/family1/28fea89f043e430b8c307e0a2d124d72 into cb6565dc6679ffd9e191e6bacbcc214d/family1 as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyPolicy/cb6565dc6679ffd9e191e6bacbcc214d/family1/24ffe0948a27475190dd3f812406b170_SeqId_46_ - updating store file list. 2023-02-11 03:54:31,864 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.HStore(882): Loaded HFile hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyPolicy/cb6565dc6679ffd9e191e6bacbcc214d/family1/24ffe0948a27475190dd3f812406b170_SeqId_46_ into cb6565dc6679ffd9e191e6bacbcc214d/family1 2023-02-11 03:54:31,864 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.HStore(855): Successfully loaded hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/staging/jenkins__testMajorCompactionWithWeeklyPolicy__8e5ngphak3rhabrhku4umtqriidrofls34u99r7r1k5v3e3ke7jen4ecohstpa72/family1/28fea89f043e430b8c307e0a2d124d72 into cb6565dc6679ffd9e191e6bacbcc214d/family1 (new location: hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyPolicy/cb6565dc6679ffd9e191e6bacbcc214d/family1/24ffe0948a27475190dd3f812406b170_SeqId_46_) 2023-02-11 03:54:31,865 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/staging/jenkins__testMajorCompactionWithWeeklyPolicy__8e5ngphak3rhabrhku4umtqriidrofls34u99r7r1k5v3e3ke7jen4ecohstpa72/family1/28fea89f043e430b8c307e0a2d124d72 2023-02-11 03:54:31,871 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMajorCompactionWithWeeklyPolicy' column='family1' 2023-02-11 03:54:31,872 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-11 03:54:31,874 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151130bbcd2930eb684aab800d930970b5e2b1 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151130bbcd2930eb684aab800d930970b5e2b1 2023-02-11 03:54:31,876 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e201512011350f34e0ab6463f9d9c438d2b4a7492 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e201512011350f34e0ab6463f9d9c438d2b4a7492 2023-02-11 03:54:31,877 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151205fa62f66f7a8344768337ac83fc6cea05 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151205fa62f66f7a8344768337ac83fc6cea05 2023-02-11 03:54:31,877 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyPolicy 3 expired mob files are deleted 2023-02-11 03:54:31,877 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130' 2023-02-11 03:54:31,877 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 3 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-02-11 03:54:31,877 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676087621935] compactions.PartitionedMobCompactor(354): After compaction, there are 5 mob files. table='testMajorCompactionWithWeeklyPolicy' column='family1' 2023-02-11 03:54:31,877 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676087621935] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyPolicy', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-11 03:54:31,878 DEBUG [PEWorker-5] locking.LockProcedure(242): UNLOCKED pid=38, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE 2023-02-11 03:54:31,880 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=38, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE in 654 msec 2023-02-11 03:54:32,018 INFO [Listener at localhost/46205] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMajorCompactionWithWeeklyPolicy Thread=267 (was 255) - Thread LEAK? -, OpenFileDescriptor=645 (was 609) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=195 (was 231), ProcessCount=168 (was 168), AvailableMemoryMB=3175 (was 3290) 2023-02-11 03:54:32,037 INFO [Listener at localhost/46205] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly Thread=267, OpenFileDescriptor=645, MaxFileDescriptor=60000, SystemLoadAverage=195, ProcessCount=168, AvailableMemoryMB=3174 2023-02-11 03:54:32,040 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] master.HMaster$4(2102): Client=jenkins//172.31.14.131 create 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} 2023-02-11 03:54:32,042 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] procedure2.ProcedureExecutor(1029): Stored pid=39, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:32,050 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=39, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly execute state=CREATE_TABLE_PRE_OPERATION 2023-02-11 03:54:32,050 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] master.MasterRpcServices(697): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly" procId is: 39 2023-02-11 03:54:32,051 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=39, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-11 03:54:32,051 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] master.MasterRpcServices(1227): Checking to see if procedure is done pid=39 2023-02-11 03:54:32,056 DEBUG [HFileArchiver-9] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/7e59b0be260b97681ec2d05527ed1bde 2023-02-11 03:54:32,057 DEBUG [HFileArchiver-9] backup.HFileArchiver(153): Directory hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/7e59b0be260b97681ec2d05527ed1bde empty. 2023-02-11 03:54:32,057 DEBUG [HFileArchiver-9] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/7e59b0be260b97681ec2d05527ed1bde 2023-02-11 03:54:32,057 DEBUG [PEWorker-2] procedure.DeleteTableProcedure(328): Archived testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly regions 2023-02-11 03:54:32,152 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] master.MasterRpcServices(1227): Checking to see if procedure is done pid=39 2023-02-11 03:54:32,353 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] master.MasterRpcServices(1227): Checking to see if procedure is done pid=39 2023-02-11 03:54:32,467 DEBUG [PEWorker-2] util.FSTableDescriptors(570): Wrote into hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/.tabledesc/.tableinfo.0000000001 2023-02-11 03:54:32,468 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(7671): creating {ENCODED => 7e59b0be260b97681ec2d05527ed1bde, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676087672040.7e59b0be260b97681ec2d05527ed1bde.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}, regionDir=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/.tmp 2023-02-11 03:54:32,588 DEBUG [RS:0;jenkins-hbase4:37011-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyPolicy/cb6565dc6679ffd9e191e6bacbcc214d/.tmp/family1/cb5686194787462e9b10f6c75d976b48 as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyPolicy/cb6565dc6679ffd9e191e6bacbcc214d/family1/cb5686194787462e9b10f6c75d976b48 2023-02-11 03:54:32,594 INFO [RS:0;jenkins-hbase4:37011-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 10 file(s) in cb6565dc6679ffd9e191e6bacbcc214d/family1 of cb6565dc6679ffd9e191e6bacbcc214d into cb5686194787462e9b10f6c75d976b48(size=6.6 K), total size for store is 337.0 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-11 03:54:32,594 DEBUG [RS:0;jenkins-hbase4:37011-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for cb6565dc6679ffd9e191e6bacbcc214d: 2023-02-11 03:54:32,594 INFO [RS:0;jenkins-hbase4:37011-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithWeeklyPolicy,,1676087661058.cb6565dc6679ffd9e191e6bacbcc214d., storeName=family1, fileCount=10, fileSize=50.0 K (5.0 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 5.5 K), priority=2, time=1676087671732; duration=0sec 2023-02-11 03:54:32,594 DEBUG [RS:0;jenkins-hbase4:37011-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-11 03:54:32,654 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] master.MasterRpcServices(1227): Checking to see if procedure is done pid=39 2023-02-11 03:54:32,697 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyPolicy/cb6565dc6679ffd9e191e6bacbcc214d/family1/365ad5fc2a874c7eac0ce82e29ac3375, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyPolicy/cb6565dc6679ffd9e191e6bacbcc214d/family1/59eab99167d44ace91e2ae21a7e01ac7, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyPolicy/cb6565dc6679ffd9e191e6bacbcc214d/family1/f2aa4ab46489463a8e3dd88b56b2ebfc, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyPolicy/cb6565dc6679ffd9e191e6bacbcc214d/family1/7a1d23c62ba24e2bbc522363c68590b6, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyPolicy/cb6565dc6679ffd9e191e6bacbcc214d/family1/1beb6a91f71047f1804d38be4477d783, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyPolicy/cb6565dc6679ffd9e191e6bacbcc214d/family1/49dd9a08fe9b433fbf1a6eec3893827e, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyPolicy/cb6565dc6679ffd9e191e6bacbcc214d/family1/6820fa5bf43945d39ae117a760887672, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyPolicy/cb6565dc6679ffd9e191e6bacbcc214d/family1/396ebaa606d9430e93f9a483a4aa749b, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyPolicy/cb6565dc6679ffd9e191e6bacbcc214d/family1/e19ff7eef6e74aec94c68ccd911bea1b, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyPolicy/cb6565dc6679ffd9e191e6bacbcc214d/family1/4a3ecd528f104618966a5aafcca91ce0_SeqId_44_] to archive 2023-02-11 03:54:32,697 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-11 03:54:32,699 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyPolicy/cb6565dc6679ffd9e191e6bacbcc214d/family1/365ad5fc2a874c7eac0ce82e29ac3375 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testMajorCompactionWithWeeklyPolicy/cb6565dc6679ffd9e191e6bacbcc214d/family1/365ad5fc2a874c7eac0ce82e29ac3375 2023-02-11 03:54:32,701 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyPolicy/cb6565dc6679ffd9e191e6bacbcc214d/family1/59eab99167d44ace91e2ae21a7e01ac7 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testMajorCompactionWithWeeklyPolicy/cb6565dc6679ffd9e191e6bacbcc214d/family1/59eab99167d44ace91e2ae21a7e01ac7 2023-02-11 03:54:32,702 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyPolicy/cb6565dc6679ffd9e191e6bacbcc214d/family1/f2aa4ab46489463a8e3dd88b56b2ebfc to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testMajorCompactionWithWeeklyPolicy/cb6565dc6679ffd9e191e6bacbcc214d/family1/f2aa4ab46489463a8e3dd88b56b2ebfc 2023-02-11 03:54:32,703 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyPolicy/cb6565dc6679ffd9e191e6bacbcc214d/family1/7a1d23c62ba24e2bbc522363c68590b6 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testMajorCompactionWithWeeklyPolicy/cb6565dc6679ffd9e191e6bacbcc214d/family1/7a1d23c62ba24e2bbc522363c68590b6 2023-02-11 03:54:32,704 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyPolicy/cb6565dc6679ffd9e191e6bacbcc214d/family1/1beb6a91f71047f1804d38be4477d783 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testMajorCompactionWithWeeklyPolicy/cb6565dc6679ffd9e191e6bacbcc214d/family1/1beb6a91f71047f1804d38be4477d783 2023-02-11 03:54:32,705 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyPolicy/cb6565dc6679ffd9e191e6bacbcc214d/family1/49dd9a08fe9b433fbf1a6eec3893827e to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testMajorCompactionWithWeeklyPolicy/cb6565dc6679ffd9e191e6bacbcc214d/family1/49dd9a08fe9b433fbf1a6eec3893827e 2023-02-11 03:54:32,706 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyPolicy/cb6565dc6679ffd9e191e6bacbcc214d/family1/6820fa5bf43945d39ae117a760887672 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testMajorCompactionWithWeeklyPolicy/cb6565dc6679ffd9e191e6bacbcc214d/family1/6820fa5bf43945d39ae117a760887672 2023-02-11 03:54:32,708 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyPolicy/cb6565dc6679ffd9e191e6bacbcc214d/family1/396ebaa606d9430e93f9a483a4aa749b to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testMajorCompactionWithWeeklyPolicy/cb6565dc6679ffd9e191e6bacbcc214d/family1/396ebaa606d9430e93f9a483a4aa749b 2023-02-11 03:54:32,709 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyPolicy/cb6565dc6679ffd9e191e6bacbcc214d/family1/e19ff7eef6e74aec94c68ccd911bea1b to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testMajorCompactionWithWeeklyPolicy/cb6565dc6679ffd9e191e6bacbcc214d/family1/e19ff7eef6e74aec94c68ccd911bea1b 2023-02-11 03:54:32,710 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyPolicy/cb6565dc6679ffd9e191e6bacbcc214d/family1/4a3ecd528f104618966a5aafcca91ce0_SeqId_44_ to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testMajorCompactionWithWeeklyPolicy/cb6565dc6679ffd9e191e6bacbcc214d/family1/4a3ecd528f104618966a5aafcca91ce0_SeqId_44_ 2023-02-11 03:54:32,875 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(865): Instantiated testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676087672040.7e59b0be260b97681ec2d05527ed1bde.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-11 03:54:32,875 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(1603): Closing 7e59b0be260b97681ec2d05527ed1bde, disabling compactions & flushes 2023-02-11 03:54:32,875 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(1625): Closing region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676087672040.7e59b0be260b97681ec2d05527ed1bde. 2023-02-11 03:54:32,875 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676087672040.7e59b0be260b97681ec2d05527ed1bde. 2023-02-11 03:54:32,875 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(1713): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676087672040.7e59b0be260b97681ec2d05527ed1bde. after waiting 0 ms 2023-02-11 03:54:32,875 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(1723): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676087672040.7e59b0be260b97681ec2d05527ed1bde. 2023-02-11 03:54:32,875 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(1837): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676087672040.7e59b0be260b97681ec2d05527ed1bde. 2023-02-11 03:54:32,875 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(1557): Region close journal for 7e59b0be260b97681ec2d05527ed1bde: Waiting for close lock at 1676087672875Disabling compacts and flushes for region at 1676087672875Disabling writes for close at 1676087672875Writing region close event to WAL at 1676087672875Closed at 1676087672875 2023-02-11 03:54:32,878 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=39, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly execute state=CREATE_TABLE_ADD_TO_META 2023-02-11 03:54:32,880 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676087672040.7e59b0be260b97681ec2d05527ed1bde.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1676087672879"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676087672879"}]},"ts":"1676087672879"} 2023-02-11 03:54:32,881 INFO [PEWorker-2] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-11 03:54:32,882 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=39, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-11 03:54:32,882 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676087672882"}]},"ts":"1676087672882"} 2023-02-11 03:54:32,884 INFO [PEWorker-2] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, state=ENABLING in hbase:meta 2023-02-11 03:54:32,888 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=40, ppid=39, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=7e59b0be260b97681ec2d05527ed1bde, ASSIGN}] 2023-02-11 03:54:32,890 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=40, ppid=39, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=7e59b0be260b97681ec2d05527ed1bde, ASSIGN 2023-02-11 03:54:32,891 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=40, ppid=39, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=7e59b0be260b97681ec2d05527ed1bde, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,37011,1676087618428; forceNewPlan=false, retain=false 2023-02-11 03:54:33,042 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=40 updating hbase:meta row=7e59b0be260b97681ec2d05527ed1bde, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:33,042 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676087672040.7e59b0be260b97681ec2d05527ed1bde.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1676087673042"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676087673042"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676087673042"}]},"ts":"1676087673042"} 2023-02-11 03:54:33,044 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=41, ppid=40, state=RUNNABLE; OpenRegionProcedure 7e59b0be260b97681ec2d05527ed1bde, server=jenkins-hbase4.apache.org,37011,1676087618428}] 2023-02-11 03:54:33,155 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] master.MasterRpcServices(1227): Checking to see if procedure is done pid=39 2023-02-11 03:54:33,198 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676087672040.7e59b0be260b97681ec2d05527ed1bde. 2023-02-11 03:54:33,199 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 7e59b0be260b97681ec2d05527ed1bde, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676087672040.7e59b0be260b97681ec2d05527ed1bde.', STARTKEY => '', ENDKEY => ''} 2023-02-11 03:54:33,199 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 7e59b0be260b97681ec2d05527ed1bde 2023-02-11 03:54:33,199 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676087672040.7e59b0be260b97681ec2d05527ed1bde.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-11 03:54:33,199 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 7e59b0be260b97681ec2d05527ed1bde 2023-02-11 03:54:33,199 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 7e59b0be260b97681ec2d05527ed1bde 2023-02-11 03:54:33,200 INFO [StoreOpener-7e59b0be260b97681ec2d05527ed1bde-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 7e59b0be260b97681ec2d05527ed1bde 2023-02-11 03:54:33,202 DEBUG [StoreOpener-7e59b0be260b97681ec2d05527ed1bde-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/7e59b0be260b97681ec2d05527ed1bde/family1 2023-02-11 03:54:33,202 DEBUG [StoreOpener-7e59b0be260b97681ec2d05527ed1bde-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/7e59b0be260b97681ec2d05527ed1bde/family1 2023-02-11 03:54:33,202 INFO [StoreOpener-7e59b0be260b97681ec2d05527ed1bde-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 7e59b0be260b97681ec2d05527ed1bde columnFamilyName family1 2023-02-11 03:54:33,204 INFO [StoreOpener-7e59b0be260b97681ec2d05527ed1bde-1] regionserver.HStore(310): Store=7e59b0be260b97681ec2d05527ed1bde/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 03:54:33,205 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/7e59b0be260b97681ec2d05527ed1bde 2023-02-11 03:54:33,205 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/7e59b0be260b97681ec2d05527ed1bde 2023-02-11 03:54:33,208 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 7e59b0be260b97681ec2d05527ed1bde 2023-02-11 03:54:33,209 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/7e59b0be260b97681ec2d05527ed1bde/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-11 03:54:33,210 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 7e59b0be260b97681ec2d05527ed1bde; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=69235474, jitterRate=0.031688958406448364}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-11 03:54:33,210 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 7e59b0be260b97681ec2d05527ed1bde: Running coprocessor pre-open hook at 1676087673199Writing region info on filesystem at 1676087673199Initializing all the Stores at 1676087673200 (+1 ms)Instantiating store for column family {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} at 1676087673200Cleaning up temporary data from old regions at 1676087673206 (+6 ms)Cleaning up detritus from prior splits at 1676087673206Running coprocessor post-open hooks at 1676087673210 (+4 ms)Region opened successfully at 1676087673210 2023-02-11 03:54:33,211 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676087672040.7e59b0be260b97681ec2d05527ed1bde., pid=41, masterSystemTime=1676087673195 2023-02-11 03:54:33,212 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676087672040.7e59b0be260b97681ec2d05527ed1bde. 2023-02-11 03:54:33,212 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676087672040.7e59b0be260b97681ec2d05527ed1bde. 2023-02-11 03:54:33,213 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=40 updating hbase:meta row=7e59b0be260b97681ec2d05527ed1bde, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:33,213 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676087672040.7e59b0be260b97681ec2d05527ed1bde.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1676087673213"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676087673213"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676087673213"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676087673213"}]},"ts":"1676087673213"} 2023-02-11 03:54:33,216 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=41, resume processing ppid=40 2023-02-11 03:54:33,216 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=41, ppid=40, state=SUCCESS; OpenRegionProcedure 7e59b0be260b97681ec2d05527ed1bde, server=jenkins-hbase4.apache.org,37011,1676087618428 in 170 msec 2023-02-11 03:54:33,218 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=40, resume processing ppid=39 2023-02-11 03:54:33,218 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=40, ppid=39, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=7e59b0be260b97681ec2d05527ed1bde, ASSIGN in 328 msec 2023-02-11 03:54:33,218 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=39, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-11 03:54:33,219 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676087673218"}]},"ts":"1676087673218"} 2023-02-11 03:54:33,220 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, state=ENABLED in hbase:meta 2023-02-11 03:54:33,222 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=39, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly execute state=CREATE_TABLE_POST_OPERATION 2023-02-11 03:54:33,224 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=39, state=SUCCESS; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly in 1.1810 sec 2023-02-11 03:54:33,961 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-02-11 03:54:34,156 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] master.MasterRpcServices(1227): Checking to see if procedure is done pid=39 2023-02-11 03:54:34,157 INFO [Listener at localhost/46205] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, procId: 39 completed 2023-02-11 03:54:34,179 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 03:54:34,181 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:34,181 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-02-11 03:54:34,181 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:54:34,181 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-02-11 03:54:34,181 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-02-11 03:54:34,181 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:34,182 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:34,183 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:34,183 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:34,183 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:34,183 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:34,183 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:34,183 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 03:54:34,183 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:34,183 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:34,184 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 03:54:34,184 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:34,184 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:34,185 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:34,185 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-02-11 03:54:34,185 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:54:34,185 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-02-11 03:54:34,185 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 03:54:34,185 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 03:54:34,185 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676087672040.7e59b0be260b97681ec2d05527ed1bde. 2023-02-11 03:54:34,185 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676087672040.7e59b0be260b97681ec2d05527ed1bde. started... 2023-02-11 03:54:34,186 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 7e59b0be260b97681ec2d05527ed1bde 1/1 column families, dataSize=74.11 KB heapSize=148.69 KB 2023-02-11 03:54:34,603 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151130eb8a06e8ba7f407db81ec47939706106 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151130eb8a06e8ba7f407db81ec47939706106 2023-02-11 03:54:35,007 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=74.1 K, hasBloomFilter=true, into tmp file hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/7e59b0be260b97681ec2d05527ed1bde/.tmp/family1/c4e5a19573734e029b5e6a9bc078519a 2023-02-11 03:54:35,012 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/7e59b0be260b97681ec2d05527ed1bde/.tmp/family1/c4e5a19573734e029b5e6a9bc078519a as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/7e59b0be260b97681ec2d05527ed1bde/family1/c4e5a19573734e029b5e6a9bc078519a 2023-02-11 03:54:35,016 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/7e59b0be260b97681ec2d05527ed1bde/family1/c4e5a19573734e029b5e6a9bc078519a, entries=1000, sequenceid=5, filesize=191.4 K 2023-02-11 03:54:35,017 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~74.11 KB/75890, heapSize ~148.67 KB/152240, currentSize=0 B/0 for 7e59b0be260b97681ec2d05527ed1bde in 831ms, sequenceid=5, compaction requested=false 2023-02-11 03:54:35,018 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 7e59b0be260b97681ec2d05527ed1bde: 2023-02-11 03:54:35,018 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676087672040.7e59b0be260b97681ec2d05527ed1bde. 2023-02-11 03:54:35,018 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 03:54:35,018 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 03:54:35,018 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:35,018 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-02-11 03:54:35,018 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37011,1676087618428' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-02-11 03:54:35,021 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:35,021 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:35,021 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:35,021 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:54:35,021 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:54:35,021 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:35,021 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 03:54:35,021 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:54:35,021 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:54:35,022 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:35,022 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:35,022 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:54:35,023 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37011,1676087618428' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-02-11 03:54:35,023 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@400f4f9a[Count = 0] remaining members to acquire global barrier 2023-02-11 03:54:35,023 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-02-11 03:54:35,023 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:35,024 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:35,024 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:35,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:35,024 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-02-11 03:54:35,024 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-02-11 03:54:35,024 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:35,024 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 03:54:35,024 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase4.apache.org,37011,1676087618428' in zk 2023-02-11 03:54:35,025 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-02-11 03:54:35,025 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:54:35,025 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:35,026 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-02-11 03:54:35,026 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:35,026 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:54:35,026 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:54:35,026 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:54:35,026 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:54:35,026 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:35,027 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:35,027 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:54:35,027 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:35,027 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:35,028 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase4.apache.org,37011,1676087618428': 2023-02-11 03:54:35,028 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37011,1676087618428' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-02-11 03:54:35,028 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-02-11 03:54:35,028 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 03:54:35,028 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 03:54:35,028 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:35,028 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 03:54:35,029 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:35,029 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:35,029 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:35,029 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:54:35,029 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:54:35,029 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:35,029 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:35,029 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:35,030 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:54:35,030 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:35,030 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:35,030 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:35,030 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:35,030 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:35,030 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:54:35,030 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:35,031 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:35,031 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:35,031 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:54:35,031 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:35,031 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:35,034 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:35,034 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:35,034 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:35,034 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:35,034 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:35,034 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:35,034 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:54:35,034 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly because node does not exist (not an error) 2023-02-11 03:54:35,034 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-02-11 03:54:35,034 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 03:54:35,034 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:35,034 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:35,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:35,034 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:35,034 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:35,035 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-02-11 03:54:35,035 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 03:54:35,035 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:35,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:35,135 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 03:54:35,136 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 03:54:35,139 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 03:54:35,141 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-02-11 03:54:35,141 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:35,141 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-02-11 03:54:35,141 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:54:35,141 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-02-11 03:54:35,141 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-02-11 03:54:35,142 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:35,142 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:35,143 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:35,143 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:35,143 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:35,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:35,143 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:35,143 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 03:54:35,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:35,143 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:35,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 03:54:35,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:35,144 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:35,144 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-02-11 03:54:35,144 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:35,144 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-02-11 03:54:35,144 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:54:35,144 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-02-11 03:54:35,144 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 03:54:35,144 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 03:54:35,144 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676087672040.7e59b0be260b97681ec2d05527ed1bde. 2023-02-11 03:54:35,144 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676087672040.7e59b0be260b97681ec2d05527ed1bde. started... 2023-02-11 03:54:35,145 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 7e59b0be260b97681ec2d05527ed1bde 1/1 column families, dataSize=74 B heapSize=408 B 2023-02-11 03:54:35,559 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151128e3a6f16ee6d04ef390cfc46c667b7d5d to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151128e3a6f16ee6d04ef390cfc46c667b7d5d 2023-02-11 03:54:35,964 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=74, hasBloomFilter=true, into tmp file hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/7e59b0be260b97681ec2d05527ed1bde/.tmp/family1/6ef47ae302584c6c900b22a4e2ff5cff 2023-02-11 03:54:35,969 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/7e59b0be260b97681ec2d05527ed1bde/.tmp/family1/6ef47ae302584c6c900b22a4e2ff5cff as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/7e59b0be260b97681ec2d05527ed1bde/family1/6ef47ae302584c6c900b22a4e2ff5cff 2023-02-11 03:54:35,973 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/7e59b0be260b97681ec2d05527ed1bde/family1/6ef47ae302584c6c900b22a4e2ff5cff, entries=1, sequenceid=9, filesize=5.0 K 2023-02-11 03:54:35,974 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~74 B/74, heapSize ~392 B/392, currentSize=0 B/0 for 7e59b0be260b97681ec2d05527ed1bde in 829ms, sequenceid=9, compaction requested=false 2023-02-11 03:54:35,974 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 7e59b0be260b97681ec2d05527ed1bde: 2023-02-11 03:54:35,974 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676087672040.7e59b0be260b97681ec2d05527ed1bde. 2023-02-11 03:54:35,974 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 03:54:35,974 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 03:54:35,974 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:35,974 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-02-11 03:54:35,974 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37011,1676087618428' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-02-11 03:54:35,976 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:35,976 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:35,976 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:35,976 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:54:35,976 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:54:35,976 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:35,976 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 03:54:35,976 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:54:35,976 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:54:35,977 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:35,977 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:35,977 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:54:35,977 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37011,1676087618428' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-02-11 03:54:35,977 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@72f72bde[Count = 0] remaining members to acquire global barrier 2023-02-11 03:54:35,977 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-02-11 03:54:35,977 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:35,979 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:35,979 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:35,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:35,979 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-02-11 03:54:35,979 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-02-11 03:54:35,979 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase4.apache.org,37011,1676087618428' in zk 2023-02-11 03:54:35,979 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:35,979 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 03:54:35,981 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:35,981 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-02-11 03:54:35,981 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:35,981 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:54:35,981 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:54:35,981 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:54:35,981 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-02-11 03:54:35,981 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:54:35,982 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:54:35,982 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:35,982 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:35,982 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:54:35,982 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:35,982 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:35,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase4.apache.org,37011,1676087618428': 2023-02-11 03:54:35,983 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37011,1676087618428' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-02-11 03:54:35,983 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-02-11 03:54:35,983 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 03:54:35,983 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 03:54:35,983 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:35,983 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 03:54:35,984 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:35,984 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:35,984 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:35,985 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:35,985 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:54:35,985 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:54:35,984 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:35,985 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:35,985 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:35,985 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:54:35,985 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:35,985 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:35,985 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:35,985 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:35,986 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:54:35,986 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:35,986 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:35,986 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:35,986 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:54:35,986 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:35,987 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:35,991 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:35,991 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:35,991 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:35,991 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:35,991 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:54:35,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:35,991 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-02-11 03:54:35,991 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 03:54:35,991 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:35,991 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:35,992 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:35,992 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:35,992 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:35,992 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-02-11 03:54:35,992 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 03:54:35,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:35,992 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:35,992 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:36,092 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 03:54:36,093 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 03:54:36,097 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 03:54:36,098 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-02-11 03:54:36,098 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:36,099 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-02-11 03:54:36,099 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:54:36,099 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-02-11 03:54:36,099 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-02-11 03:54:36,099 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:36,099 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:36,100 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:36,100 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:36,100 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:36,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:36,101 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:36,101 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 03:54:36,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:36,101 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:36,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 03:54:36,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:36,101 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:36,102 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-02-11 03:54:36,102 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:36,102 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-02-11 03:54:36,102 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:54:36,102 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-02-11 03:54:36,102 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 03:54:36,102 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 03:54:36,102 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676087672040.7e59b0be260b97681ec2d05527ed1bde. 2023-02-11 03:54:36,102 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676087672040.7e59b0be260b97681ec2d05527ed1bde. started... 2023-02-11 03:54:36,102 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 7e59b0be260b97681ec2d05527ed1bde 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-11 03:54:36,514 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512013c311260cdaa4a179c410de6ce59854e to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201512013c311260cdaa4a179c410de6ce59854e 2023-02-11 03:54:36,919 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/7e59b0be260b97681ec2d05527ed1bde/.tmp/family1/08ac5f1d679041beb01fc7926f1c1233 2023-02-11 03:54:36,924 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/7e59b0be260b97681ec2d05527ed1bde/.tmp/family1/08ac5f1d679041beb01fc7926f1c1233 as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/7e59b0be260b97681ec2d05527ed1bde/family1/08ac5f1d679041beb01fc7926f1c1233 2023-02-11 03:54:36,928 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/7e59b0be260b97681ec2d05527ed1bde/family1/08ac5f1d679041beb01fc7926f1c1233, entries=1, sequenceid=13, filesize=5.0 K 2023-02-11 03:54:36,929 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 7e59b0be260b97681ec2d05527ed1bde in 827ms, sequenceid=13, compaction requested=true 2023-02-11 03:54:36,929 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 7e59b0be260b97681ec2d05527ed1bde: 2023-02-11 03:54:36,929 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676087672040.7e59b0be260b97681ec2d05527ed1bde. 2023-02-11 03:54:36,929 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 03:54:36,929 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 03:54:36,930 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:36,930 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-02-11 03:54:36,930 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37011,1676087618428' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-02-11 03:54:36,931 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:36,931 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:36,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:36,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:54:36,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:54:36,931 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:36,931 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 03:54:36,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:54:36,932 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:54:36,932 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:36,932 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:36,932 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:54:36,933 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37011,1676087618428' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-02-11 03:54:36,933 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@50b4fbe2[Count = 0] remaining members to acquire global barrier 2023-02-11 03:54:36,933 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-02-11 03:54:36,933 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:36,934 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:36,934 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:36,934 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:36,934 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-02-11 03:54:36,934 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-02-11 03:54:36,934 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase4.apache.org,37011,1676087618428' in zk 2023-02-11 03:54:36,934 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:36,934 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 03:54:36,936 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:36,936 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-02-11 03:54:36,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:36,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:54:36,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:54:36,936 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:54:36,936 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-02-11 03:54:36,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:54:36,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:54:36,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:36,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:36,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:54:36,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:36,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:36,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase4.apache.org,37011,1676087618428': 2023-02-11 03:54:36,939 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37011,1676087618428' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-02-11 03:54:36,939 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-02-11 03:54:36,939 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 03:54:36,939 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 03:54:36,939 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:36,939 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 03:54:36,940 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:36,940 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:36,940 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:36,940 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:54:36,940 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:54:36,940 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:36,940 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:36,940 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:36,941 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:54:36,941 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:36,941 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:36,941 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:36,941 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:36,941 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:36,941 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:54:36,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:36,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:36,942 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:36,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:54:36,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:36,943 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:36,944 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:36,944 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:36,944 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:36,944 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:36,944 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:36,944 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:36,944 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-02-11 03:54:36,944 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 03:54:36,944 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:54:36,944 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:36,944 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:36,944 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:36,944 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:36,945 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:36,945 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-02-11 03:54:36,945 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 03:54:36,945 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:36,945 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:37,045 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 03:54:37,046 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 03:54:37,050 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 03:54:37,051 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-02-11 03:54:37,051 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:37,051 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-02-11 03:54:37,051 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:54:37,051 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-02-11 03:54:37,051 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-02-11 03:54:37,052 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:37,052 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:37,053 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:37,053 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:37,053 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:37,053 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:37,053 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:37,053 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 03:54:37,054 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:37,054 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:37,054 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 03:54:37,054 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:37,054 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:37,054 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-02-11 03:54:37,054 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:37,054 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-02-11 03:54:37,054 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:54:37,054 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-02-11 03:54:37,055 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 03:54:37,055 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 03:54:37,055 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676087672040.7e59b0be260b97681ec2d05527ed1bde. 2023-02-11 03:54:37,055 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676087672040.7e59b0be260b97681ec2d05527ed1bde. started... 2023-02-11 03:54:37,055 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 7e59b0be260b97681ec2d05527ed1bde 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-11 03:54:37,467 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151205eb8f1ddd83834018a712ccf29db5da76 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151205eb8f1ddd83834018a712ccf29db5da76 2023-02-11 03:54:37,872 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/7e59b0be260b97681ec2d05527ed1bde/.tmp/family1/94e9c6c9026042d386f284b3000d27bd 2023-02-11 03:54:37,877 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/7e59b0be260b97681ec2d05527ed1bde/.tmp/family1/94e9c6c9026042d386f284b3000d27bd as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/7e59b0be260b97681ec2d05527ed1bde/family1/94e9c6c9026042d386f284b3000d27bd 2023-02-11 03:54:37,882 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/7e59b0be260b97681ec2d05527ed1bde/family1/94e9c6c9026042d386f284b3000d27bd, entries=1, sequenceid=17, filesize=5.0 K 2023-02-11 03:54:37,882 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 7e59b0be260b97681ec2d05527ed1bde in 827ms, sequenceid=17, compaction requested=true 2023-02-11 03:54:37,882 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 7e59b0be260b97681ec2d05527ed1bde: 2023-02-11 03:54:37,882 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676087672040.7e59b0be260b97681ec2d05527ed1bde. 2023-02-11 03:54:37,883 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 03:54:37,883 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 03:54:37,883 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:37,883 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-02-11 03:54:37,883 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37011,1676087618428' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-02-11 03:54:37,885 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:37,885 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:37,885 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:37,886 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:54:37,886 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:54:37,886 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:37,886 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 03:54:37,886 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:54:37,886 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:54:37,886 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:37,886 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:37,887 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:54:37,887 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37011,1676087618428' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-02-11 03:54:37,887 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@53e77bf[Count = 0] remaining members to acquire global barrier 2023-02-11 03:54:37,887 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-02-11 03:54:37,887 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:37,888 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:37,888 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:37,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:37,888 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-02-11 03:54:37,888 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-02-11 03:54:37,888 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:37,888 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 03:54:37,888 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase4.apache.org,37011,1676087618428' in zk 2023-02-11 03:54:37,890 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-02-11 03:54:37,890 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:37,890 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:54:37,890 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:37,890 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:54:37,890 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:54:37,890 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-02-11 03:54:37,890 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:54:37,890 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:54:37,891 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:37,891 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:37,891 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:54:37,891 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:37,892 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:37,892 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase4.apache.org,37011,1676087618428': 2023-02-11 03:54:37,892 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37011,1676087618428' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-02-11 03:54:37,892 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-02-11 03:54:37,892 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 03:54:37,892 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 03:54:37,892 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:37,892 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 03:54:37,893 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:37,894 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:37,894 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:37,894 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:54:37,894 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:54:37,894 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:37,894 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:37,894 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:37,894 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:54:37,894 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:37,894 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:37,894 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:37,894 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:37,894 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:37,894 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:54:37,895 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:37,895 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:37,895 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:37,895 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:54:37,896 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:37,896 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:37,898 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:37,898 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:37,898 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:37,898 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:37,899 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:54:37,899 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly because node does not exist (not an error) 2023-02-11 03:54:37,899 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:37,899 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:37,899 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:37,899 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-02-11 03:54:37,899 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 03:54:37,899 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:37,899 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:37,899 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:37,899 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:37,899 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:37,899 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-02-11 03:54:37,899 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 03:54:37,899 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:37,999 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 03:54:38,000 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 03:54:38,005 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 03:54:38,006 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-02-11 03:54:38,006 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:38,006 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-02-11 03:54:38,006 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:54:38,006 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-02-11 03:54:38,006 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-02-11 03:54:38,007 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:38,007 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:38,008 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:38,008 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:38,008 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:38,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:38,009 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:38,009 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 03:54:38,009 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:38,009 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:38,009 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 03:54:38,009 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:38,009 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:38,010 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-02-11 03:54:38,010 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:38,010 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-02-11 03:54:38,010 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:54:38,010 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-02-11 03:54:38,012 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 03:54:38,012 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 03:54:38,012 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676087672040.7e59b0be260b97681ec2d05527ed1bde. 2023-02-11 03:54:38,012 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676087672040.7e59b0be260b97681ec2d05527ed1bde. started... 2023-02-11 03:54:38,012 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 7e59b0be260b97681ec2d05527ed1bde 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-11 03:54:38,236 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-02-11 03:54:38,236 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x70415025 to 127.0.0.1:53383 2023-02-11 03:54:38,236 DEBUG [Finalizer] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-11 03:54:38,236 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-02-11 03:54:38,237 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x307d34e9 to 127.0.0.1:53383 2023-02-11 03:54:38,237 DEBUG [Finalizer] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-11 03:54:38,237 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-02-11 03:54:38,237 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x386dac4f to 127.0.0.1:53383 2023-02-11 03:54:38,237 DEBUG [Finalizer] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-11 03:54:38,237 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-02-11 03:54:38,237 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x641abd90 to 127.0.0.1:53383 2023-02-11 03:54:38,237 DEBUG [Finalizer] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-11 03:54:38,426 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151228da0124039cda4f05a1b53d0b6deb305c to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151228da0124039cda4f05a1b53d0b6deb305c 2023-02-11 03:54:38,754 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-11 03:54:38,831 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=21, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/7e59b0be260b97681ec2d05527ed1bde/.tmp/family1/e1c919793ac34ed08250216d47286715 2023-02-11 03:54:38,836 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/7e59b0be260b97681ec2d05527ed1bde/.tmp/family1/e1c919793ac34ed08250216d47286715 as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/7e59b0be260b97681ec2d05527ed1bde/family1/e1c919793ac34ed08250216d47286715 2023-02-11 03:54:38,840 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/7e59b0be260b97681ec2d05527ed1bde/family1/e1c919793ac34ed08250216d47286715, entries=1, sequenceid=21, filesize=5.0 K 2023-02-11 03:54:38,840 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 7e59b0be260b97681ec2d05527ed1bde in 828ms, sequenceid=21, compaction requested=true 2023-02-11 03:54:38,841 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 7e59b0be260b97681ec2d05527ed1bde: 2023-02-11 03:54:38,841 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676087672040.7e59b0be260b97681ec2d05527ed1bde. 2023-02-11 03:54:38,841 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 03:54:38,841 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 03:54:38,841 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:38,841 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-02-11 03:54:38,841 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37011,1676087618428' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-02-11 03:54:38,846 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:38,846 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:38,846 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:38,846 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:54:38,846 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:54:38,846 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:38,846 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 03:54:38,846 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:54:38,846 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:54:38,846 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:38,847 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:38,847 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:54:38,847 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37011,1676087618428' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-02-11 03:54:38,847 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-02-11 03:54:38,847 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3f779b1d[Count = 0] remaining members to acquire global barrier 2023-02-11 03:54:38,847 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:38,848 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:38,848 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:38,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:38,848 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-02-11 03:54:38,849 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-02-11 03:54:38,848 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:38,849 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 03:54:38,849 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase4.apache.org,37011,1676087618428' in zk 2023-02-11 03:54:38,852 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-02-11 03:54:38,852 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:54:38,852 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:38,853 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-02-11 03:54:38,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:38,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:54:38,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:54:38,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:54:38,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:54:38,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:38,854 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:38,854 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:54:38,854 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:38,854 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:38,854 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase4.apache.org,37011,1676087618428': 2023-02-11 03:54:38,854 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37011,1676087618428' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-02-11 03:54:38,854 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-02-11 03:54:38,855 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 03:54:38,855 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 03:54:38,855 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:38,855 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 03:54:38,856 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:38,856 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:38,856 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:38,856 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:38,856 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:38,856 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:54:38,856 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:54:38,856 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:38,856 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:38,856 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:38,856 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:54:38,857 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:38,857 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:38,857 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:38,857 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:54:38,857 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:38,858 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:38,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:38,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:54:38,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:38,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:38,859 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:38,859 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:38,860 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:38,860 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:38,860 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:38,860 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:38,860 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-02-11 03:54:38,860 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly because node does not exist (not an error) 2023-02-11 03:54:38,860 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:38,860 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 03:54:38,860 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:38,860 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:54:38,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:38,860 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:38,860 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-02-11 03:54:38,860 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:38,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:38,860 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:38,860 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 03:54:38,961 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 03:54:38,961 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 03:54:38,965 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 03:54:38,967 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-02-11 03:54:38,967 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:38,967 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-02-11 03:54:38,967 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:54:38,967 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-02-11 03:54:38,967 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-02-11 03:54:38,968 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:38,968 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:38,969 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:38,969 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:38,969 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:38,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:38,969 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:38,969 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 03:54:38,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:38,969 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:38,970 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 03:54:38,970 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:38,970 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:38,970 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-02-11 03:54:38,970 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:38,970 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-02-11 03:54:38,970 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:54:38,970 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-02-11 03:54:38,970 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 03:54:38,970 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 03:54:38,971 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676087672040.7e59b0be260b97681ec2d05527ed1bde. 2023-02-11 03:54:38,971 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676087672040.7e59b0be260b97681ec2d05527ed1bde. started... 2023-02-11 03:54:38,971 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 7e59b0be260b97681ec2d05527ed1bde 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-11 03:54:39,385 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151231a6db5f18b2ea4f059ea21ba788abe5aa to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151231a6db5f18b2ea4f059ea21ba788abe5aa 2023-02-11 03:54:39,791 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=25, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/7e59b0be260b97681ec2d05527ed1bde/.tmp/family1/19bc8225da374841a0584d285026c93b 2023-02-11 03:54:39,795 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/7e59b0be260b97681ec2d05527ed1bde/.tmp/family1/19bc8225da374841a0584d285026c93b as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/7e59b0be260b97681ec2d05527ed1bde/family1/19bc8225da374841a0584d285026c93b 2023-02-11 03:54:39,799 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/7e59b0be260b97681ec2d05527ed1bde/family1/19bc8225da374841a0584d285026c93b, entries=1, sequenceid=25, filesize=5.0 K 2023-02-11 03:54:39,800 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 7e59b0be260b97681ec2d05527ed1bde in 829ms, sequenceid=25, compaction requested=true 2023-02-11 03:54:39,800 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 7e59b0be260b97681ec2d05527ed1bde: 2023-02-11 03:54:39,800 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676087672040.7e59b0be260b97681ec2d05527ed1bde. 2023-02-11 03:54:39,800 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 03:54:39,800 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 03:54:39,800 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:39,800 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-02-11 03:54:39,800 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37011,1676087618428' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-02-11 03:54:39,801 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:39,801 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:39,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:39,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:54:39,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:54:39,802 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:39,802 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 03:54:39,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:54:39,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:54:39,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:39,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:39,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:54:39,803 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37011,1676087618428' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-02-11 03:54:39,803 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7415eef6[Count = 0] remaining members to acquire global barrier 2023-02-11 03:54:39,803 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-02-11 03:54:39,803 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:39,805 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:39,805 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:39,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:39,806 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-02-11 03:54:39,806 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-02-11 03:54:39,806 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:39,806 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 03:54:39,806 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase4.apache.org,37011,1676087618428' in zk 2023-02-11 03:54:39,807 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:39,807 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-02-11 03:54:39,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:39,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:54:39,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:54:39,807 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:54:39,807 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-02-11 03:54:39,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:54:39,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:54:39,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:39,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:39,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:54:39,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:39,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:39,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase4.apache.org,37011,1676087618428': 2023-02-11 03:54:39,809 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37011,1676087618428' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-02-11 03:54:39,809 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-02-11 03:54:39,809 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 03:54:39,809 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 03:54:39,809 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:39,809 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 03:54:39,810 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:39,811 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:39,811 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:39,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:39,811 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:39,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:39,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:54:39,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:54:39,811 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:39,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:39,811 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:39,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:54:39,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:39,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:39,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:54:39,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:39,812 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:39,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:39,813 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:54:39,813 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:39,813 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:39,814 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:39,814 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:39,814 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:39,814 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:54:39,814 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:39,814 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-02-11 03:54:39,814 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:39,814 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:39,814 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:39,814 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:39,814 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 03:54:39,814 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:39,815 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:39,815 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly because node does not exist (not an error) 2023-02-11 03:54:39,815 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:39,815 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:39,815 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:39,815 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-02-11 03:54:39,815 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 03:54:39,915 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 03:54:39,916 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 03:54:39,919 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 03:54:39,921 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-02-11 03:54:39,921 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:39,921 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-02-11 03:54:39,921 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:54:39,921 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-02-11 03:54:39,921 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-02-11 03:54:39,921 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:39,921 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:39,924 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:39,924 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:39,924 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:39,924 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:39,924 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:39,924 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 03:54:39,924 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:39,925 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:39,925 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 03:54:39,925 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:39,925 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:39,925 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-02-11 03:54:39,925 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:39,925 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-02-11 03:54:39,925 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:54:39,925 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-02-11 03:54:39,926 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 03:54:39,926 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 03:54:39,926 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676087672040.7e59b0be260b97681ec2d05527ed1bde. 2023-02-11 03:54:39,926 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676087672040.7e59b0be260b97681ec2d05527ed1bde. started... 2023-02-11 03:54:39,926 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 7e59b0be260b97681ec2d05527ed1bde 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-11 03:54:40,338 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160101d5a8a5a3318041d7bede4114aec8652a to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20160101d5a8a5a3318041d7bede4114aec8652a 2023-02-11 03:54:40,744 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=29, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/7e59b0be260b97681ec2d05527ed1bde/.tmp/family1/bfbba689fa2d4c129ad1ca9e24762cd2 2023-02-11 03:54:40,749 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/7e59b0be260b97681ec2d05527ed1bde/.tmp/family1/bfbba689fa2d4c129ad1ca9e24762cd2 as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/7e59b0be260b97681ec2d05527ed1bde/family1/bfbba689fa2d4c129ad1ca9e24762cd2 2023-02-11 03:54:40,752 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/7e59b0be260b97681ec2d05527ed1bde/family1/bfbba689fa2d4c129ad1ca9e24762cd2, entries=1, sequenceid=29, filesize=5.0 K 2023-02-11 03:54:40,753 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 7e59b0be260b97681ec2d05527ed1bde in 827ms, sequenceid=29, compaction requested=true 2023-02-11 03:54:40,753 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 7e59b0be260b97681ec2d05527ed1bde: 2023-02-11 03:54:40,753 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676087672040.7e59b0be260b97681ec2d05527ed1bde. 2023-02-11 03:54:40,753 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 03:54:40,753 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 03:54:40,753 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:40,753 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-02-11 03:54:40,753 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37011,1676087618428' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-02-11 03:54:40,755 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:40,755 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:40,755 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:40,755 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:54:40,755 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:54:40,755 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:40,755 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 03:54:40,755 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:54:40,755 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:54:40,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:40,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:40,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:54:40,756 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37011,1676087618428' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-02-11 03:54:40,756 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@309707d[Count = 0] remaining members to acquire global barrier 2023-02-11 03:54:40,756 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-02-11 03:54:40,756 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:40,757 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:40,757 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:40,757 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:40,757 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-02-11 03:54:40,758 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-02-11 03:54:40,758 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase4.apache.org,37011,1676087618428' in zk 2023-02-11 03:54:40,758 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:40,758 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 03:54:40,760 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-02-11 03:54:40,760 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:54:40,760 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:40,760 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-02-11 03:54:40,760 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:40,760 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:54:40,760 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:54:40,760 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:54:40,760 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:54:40,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:40,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:40,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:54:40,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:40,762 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:40,762 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase4.apache.org,37011,1676087618428': 2023-02-11 03:54:40,762 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37011,1676087618428' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-02-11 03:54:40,762 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-02-11 03:54:40,762 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 03:54:40,762 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 03:54:40,762 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:40,762 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 03:54:40,767 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:40,767 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:40,767 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:40,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:40,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:54:40,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:54:40,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:40,767 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:40,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:54:40,767 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:40,767 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:40,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:40,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:40,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:40,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:54:40,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:40,768 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:40,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:40,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:54:40,769 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:40,769 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:40,770 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:40,770 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:40,770 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:40,770 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:40,770 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:40,770 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-02-11 03:54:40,770 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 03:54:40,770 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:40,770 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:40,770 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:40,770 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly because node does not exist (not an error) 2023-02-11 03:54:40,770 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:40,770 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:54:40,770 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:40,770 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:40,770 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-02-11 03:54:40,771 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 03:54:40,771 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:40,771 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:40,871 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 03:54:40,871 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 03:54:40,875 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 03:54:40,877 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-02-11 03:54:40,877 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:40,877 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-02-11 03:54:40,877 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:54:40,877 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-02-11 03:54:40,877 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-02-11 03:54:40,877 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:40,877 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:40,879 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:40,879 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:40,879 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:40,880 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:40,880 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:40,880 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 03:54:40,880 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:40,880 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:40,880 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 03:54:40,880 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:40,880 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:40,881 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-02-11 03:54:40,881 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:40,881 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-02-11 03:54:40,881 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:54:40,881 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-02-11 03:54:40,881 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 03:54:40,881 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 03:54:40,881 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676087672040.7e59b0be260b97681ec2d05527ed1bde. 2023-02-11 03:54:40,881 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676087672040.7e59b0be260b97681ec2d05527ed1bde. started... 2023-02-11 03:54:40,881 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 7e59b0be260b97681ec2d05527ed1bde 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-11 03:54:41,294 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201601038dbf50b4a40d453ba2a8261a2e5741af to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201601038dbf50b4a40d453ba2a8261a2e5741af 2023-02-11 03:54:41,699 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=33, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/7e59b0be260b97681ec2d05527ed1bde/.tmp/family1/8203077f03534604846eb7a86101fc21 2023-02-11 03:54:41,704 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/7e59b0be260b97681ec2d05527ed1bde/.tmp/family1/8203077f03534604846eb7a86101fc21 as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/7e59b0be260b97681ec2d05527ed1bde/family1/8203077f03534604846eb7a86101fc21 2023-02-11 03:54:41,708 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/7e59b0be260b97681ec2d05527ed1bde/family1/8203077f03534604846eb7a86101fc21, entries=1, sequenceid=33, filesize=5.0 K 2023-02-11 03:54:41,709 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 7e59b0be260b97681ec2d05527ed1bde in 828ms, sequenceid=33, compaction requested=true 2023-02-11 03:54:41,709 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 7e59b0be260b97681ec2d05527ed1bde: 2023-02-11 03:54:41,709 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676087672040.7e59b0be260b97681ec2d05527ed1bde. 2023-02-11 03:54:41,709 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 03:54:41,709 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 03:54:41,709 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:41,709 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-02-11 03:54:41,709 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37011,1676087618428' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-02-11 03:54:41,710 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:41,710 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:41,710 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:41,711 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:54:41,711 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:54:41,711 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:41,711 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 03:54:41,711 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:54:41,711 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:54:41,711 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:41,711 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:41,712 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:54:41,712 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37011,1676087618428' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-02-11 03:54:41,712 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@757c6c2c[Count = 0] remaining members to acquire global barrier 2023-02-11 03:54:41,712 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-02-11 03:54:41,712 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:41,713 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:41,713 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:41,713 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:41,713 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-02-11 03:54:41,713 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-02-11 03:54:41,713 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase4.apache.org,37011,1676087618428' in zk 2023-02-11 03:54:41,713 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:41,713 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 03:54:41,714 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:41,714 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-02-11 03:54:41,714 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:54:41,714 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:41,714 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:54:41,714 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:54:41,714 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-02-11 03:54:41,715 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:54:41,715 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:54:41,715 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:41,715 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:41,716 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:54:41,716 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:41,716 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:41,716 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase4.apache.org,37011,1676087618428': 2023-02-11 03:54:41,716 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37011,1676087618428' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-02-11 03:54:41,716 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-02-11 03:54:41,716 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 03:54:41,716 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 03:54:41,717 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:41,717 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 03:54:41,719 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:41,719 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:41,719 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:41,719 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:54:41,719 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:54:41,719 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:41,719 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:41,719 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:41,720 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:54:41,720 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:41,720 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:41,720 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:41,720 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:41,720 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:41,720 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:54:41,721 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:41,721 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:41,721 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:41,721 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:54:41,721 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:41,722 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:41,722 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:41,722 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:41,722 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:41,723 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:54:41,723 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-02-11 03:54:41,723 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 03:54:41,722 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:41,723 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:41,723 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:41,723 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:41,723 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:41,723 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:41,723 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-02-11 03:54:41,723 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 03:54:41,723 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:41,723 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly because node does not exist (not an error) 2023-02-11 03:54:41,723 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:41,723 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:41,723 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:41,823 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 03:54:41,824 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 03:54:41,828 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 03:54:41,829 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-02-11 03:54:41,829 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:41,829 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-02-11 03:54:41,829 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:54:41,829 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-02-11 03:54:41,829 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-02-11 03:54:41,830 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:41,830 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:41,831 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:41,831 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:41,831 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:41,831 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:41,831 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:41,831 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 03:54:41,831 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:41,831 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:41,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 03:54:41,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:41,832 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:41,832 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-02-11 03:54:41,832 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:41,832 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-02-11 03:54:41,832 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:54:41,832 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-02-11 03:54:41,832 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 03:54:41,832 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 03:54:41,832 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676087672040.7e59b0be260b97681ec2d05527ed1bde. 2023-02-11 03:54:41,832 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676087672040.7e59b0be260b97681ec2d05527ed1bde. started... 2023-02-11 03:54:41,832 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 7e59b0be260b97681ec2d05527ed1bde 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-11 03:54:42,245 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20150907dbf03e1cc2f349328d4dbd1161739361 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20150907dbf03e1cc2f349328d4dbd1161739361 2023-02-11 03:54:42,650 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=37, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/7e59b0be260b97681ec2d05527ed1bde/.tmp/family1/f7e01afc549f44c9b55237ba2f97fe5d 2023-02-11 03:54:42,654 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/7e59b0be260b97681ec2d05527ed1bde/.tmp/family1/f7e01afc549f44c9b55237ba2f97fe5d as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/7e59b0be260b97681ec2d05527ed1bde/family1/f7e01afc549f44c9b55237ba2f97fe5d 2023-02-11 03:54:42,658 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/7e59b0be260b97681ec2d05527ed1bde/family1/f7e01afc549f44c9b55237ba2f97fe5d, entries=1, sequenceid=37, filesize=5.0 K 2023-02-11 03:54:42,659 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 7e59b0be260b97681ec2d05527ed1bde in 827ms, sequenceid=37, compaction requested=true 2023-02-11 03:54:42,659 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 7e59b0be260b97681ec2d05527ed1bde: 2023-02-11 03:54:42,659 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676087672040.7e59b0be260b97681ec2d05527ed1bde. 2023-02-11 03:54:42,659 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 03:54:42,659 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 03:54:42,659 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:42,659 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-02-11 03:54:42,659 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37011,1676087618428' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-02-11 03:54:42,660 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:42,660 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:42,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:42,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:54:42,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:54:42,661 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:42,661 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 03:54:42,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:54:42,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:54:42,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:42,662 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:42,662 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:54:42,662 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37011,1676087618428' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-02-11 03:54:42,662 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7650beed[Count = 0] remaining members to acquire global barrier 2023-02-11 03:54:42,662 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-02-11 03:54:42,662 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:42,664 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:42,664 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:42,664 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:42,664 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-02-11 03:54:42,664 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-02-11 03:54:42,664 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:42,664 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 03:54:42,664 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase4.apache.org,37011,1676087618428' in zk 2023-02-11 03:54:42,665 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-02-11 03:54:42,665 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:54:42,665 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:42,665 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-02-11 03:54:42,665 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:42,665 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:54:42,665 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:54:42,666 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:54:42,666 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:54:42,666 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:42,666 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:42,667 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:54:42,667 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:42,667 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:42,668 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase4.apache.org,37011,1676087618428': 2023-02-11 03:54:42,668 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37011,1676087618428' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-02-11 03:54:42,668 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-02-11 03:54:42,668 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 03:54:42,668 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 03:54:42,668 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:42,668 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 03:54:42,669 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:42,669 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:42,669 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:42,669 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:42,669 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:54:42,669 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:54:42,669 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:42,669 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:42,669 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:54:42,669 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:42,669 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:42,670 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:42,670 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:42,670 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:42,670 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:54:42,670 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:42,670 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:42,671 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:42,671 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:54:42,671 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:42,671 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:42,676 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:42,676 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:42,676 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:42,676 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:42,677 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly because node does not exist (not an error) 2023-02-11 03:54:42,677 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:42,676 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-02-11 03:54:42,677 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 03:54:42,676 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:42,676 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:54:42,676 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:42,677 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:42,677 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:42,677 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:42,677 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:42,677 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:42,677 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-02-11 03:54:42,677 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 03:54:42,677 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:42,777 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 03:54:42,778 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 03:54:42,782 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 03:54:42,784 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-02-11 03:54:42,784 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:42,784 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-02-11 03:54:42,784 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:54:42,784 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-02-11 03:54:42,784 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-02-11 03:54:42,784 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:42,784 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:42,785 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:42,785 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:42,785 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:42,785 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:42,786 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:42,786 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 03:54:42,786 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:42,786 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:42,786 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 03:54:42,786 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:42,786 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:42,787 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-02-11 03:54:42,787 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:42,787 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-02-11 03:54:42,787 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:54:42,787 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-02-11 03:54:42,790 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 03:54:42,790 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 03:54:42,790 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676087672040.7e59b0be260b97681ec2d05527ed1bde. 2023-02-11 03:54:42,790 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676087672040.7e59b0be260b97681ec2d05527ed1bde. started... 2023-02-11 03:54:42,790 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 7e59b0be260b97681ec2d05527ed1bde 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-11 03:54:43,202 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201511206d60b9c2ce0b4ff7b5b20982466710f6 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201511206d60b9c2ce0b4ff7b5b20982466710f6 2023-02-11 03:54:43,607 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=41, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/7e59b0be260b97681ec2d05527ed1bde/.tmp/family1/f68c7a9e88ed43839ce25eeee70560d6 2023-02-11 03:54:43,611 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/7e59b0be260b97681ec2d05527ed1bde/.tmp/family1/f68c7a9e88ed43839ce25eeee70560d6 as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/7e59b0be260b97681ec2d05527ed1bde/family1/f68c7a9e88ed43839ce25eeee70560d6 2023-02-11 03:54:43,615 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/7e59b0be260b97681ec2d05527ed1bde/family1/f68c7a9e88ed43839ce25eeee70560d6, entries=1, sequenceid=41, filesize=5.0 K 2023-02-11 03:54:43,616 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 7e59b0be260b97681ec2d05527ed1bde in 826ms, sequenceid=41, compaction requested=true 2023-02-11 03:54:43,616 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 7e59b0be260b97681ec2d05527ed1bde: 2023-02-11 03:54:43,616 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676087672040.7e59b0be260b97681ec2d05527ed1bde. 2023-02-11 03:54:43,616 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 03:54:43,616 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 03:54:43,616 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:43,616 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-02-11 03:54:43,616 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37011,1676087618428' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-02-11 03:54:43,618 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:43,618 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:43,618 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:43,618 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:54:43,618 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:54:43,618 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:43,618 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:54:43,618 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 03:54:43,618 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:54:43,619 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:43,619 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:43,619 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:54:43,619 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37011,1676087618428' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-02-11 03:54:43,619 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2d699c8f[Count = 0] remaining members to acquire global barrier 2023-02-11 03:54:43,619 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-02-11 03:54:43,619 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:43,621 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:43,621 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:43,621 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:43,622 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-02-11 03:54:43,622 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:43,622 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 03:54:43,622 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-02-11 03:54:43,622 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase4.apache.org,37011,1676087618428' in zk 2023-02-11 03:54:43,623 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:43,623 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-02-11 03:54:43,623 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:43,623 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:54:43,623 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:54:43,623 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:54:43,623 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-02-11 03:54:43,623 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:54:43,623 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:54:43,624 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:43,624 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:43,624 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:54:43,624 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:43,624 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:43,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase4.apache.org,37011,1676087618428': 2023-02-11 03:54:43,625 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37011,1676087618428' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-02-11 03:54:43,625 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-02-11 03:54:43,625 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 03:54:43,625 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 03:54:43,625 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:43,625 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 03:54:43,627 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:43,627 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:43,627 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:43,627 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:54:43,627 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:54:43,627 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:43,627 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:43,627 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:43,627 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:54:43,627 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:43,627 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:43,627 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:43,627 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:43,627 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:43,628 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:54:43,628 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:43,628 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:43,628 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:43,628 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:54:43,629 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:43,629 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:43,630 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:43,630 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:43,630 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:43,630 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:43,630 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:43,630 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-02-11 03:54:43,630 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly because node does not exist (not an error) 2023-02-11 03:54:43,630 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:43,630 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 03:54:43,630 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:54:43,630 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:43,630 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:43,630 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:43,631 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:43,631 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:43,631 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:43,631 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-02-11 03:54:43,631 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 03:54:43,631 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:43,731 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 03:54:43,732 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 03:54:43,733 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=39921] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}] of the table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:43,734 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676087621935] procedure2.ProcedureExecutor(1029): Stored pid=42, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-02-11 03:54:43,735 DEBUG [PEWorker-3] locking.LockProcedure(309): LOCKED pid=42, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-02-11 03:54:43,736 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676087621935] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-11 03:54:43,736 INFO [PEWorker-3] procedure2.TimeoutExecutorThread(81): ADDED pid=42, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1676088283736 2023-02-11 03:54:43,737 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676087621935] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-02-11 03:54:43,738 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676087621935] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 10 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column 'family1' 2023-02-11 03:54:43,738 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676087621935] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-02-11 03:54:43,739 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676087621935] zookeeper.ReadOnlyZKClient(139): Connect 0x79fa0667 to 127.0.0.1:53383 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-11 03:54:43,743 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676087621935] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@2cd6b9de, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-11 03:54:43,743 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-11 03:54:44,578 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160103a735afe951e84f2d9aff2f7286f488a2 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20160103a735afe951e84f2d9aff2f7286f488a2 2023-02-11 03:54:44,580 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-02-11 03:54:44,592 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-11 03:54:44,593 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:44462, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-11 03:54:44,599 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-02-11 03:54:44,601 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:54204, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-02-11 03:54:44,609 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151228/family1/2cdc03c039b34093ae25f40bb029a290 first=Optional[r3] last=Optional[r6] 2023-02-11 03:54:44,610 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676087672040.7e59b0be260b97681ec2d05527ed1bde., hostname=jenkins-hbase4.apache.org,37011,1676087618428, seqNum=2 for row with hfile group [{family1,hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151228/family1/2cdc03c039b34093ae25f40bb029a290}] 2023-02-11 03:54:44,614 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.HStore(763): Validating hfile at hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151228/family1/2cdc03c039b34093ae25f40bb029a290 for inclusion in 7e59b0be260b97681ec2d05527ed1bde/family1 2023-02-11 03:54:44,616 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.HStore(775): HFile bounds: first=r3 last=r6 2023-02-11 03:54:44,616 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.HStore(777): Region bounds: first= last= 2023-02-11 03:54:44,616 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.HRegion(2445): Flush status journal for 7e59b0be260b97681ec2d05527ed1bde: 2023-02-11 03:54:44,617 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151228/family1/2cdc03c039b34093ae25f40bb029a290 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__d13c0ds831vdl4tqoa44hkr607cmcvn47ieojlrniipli0iob85gk7lkvjtnkbq2/family1/2cdc03c039b34093ae25f40bb029a290 2023-02-11 03:54:44,619 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__d13c0ds831vdl4tqoa44hkr607cmcvn47ieojlrniipli0iob85gk7lkvjtnkbq2/family1/2cdc03c039b34093ae25f40bb029a290 as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/7e59b0be260b97681ec2d05527ed1bde/family1/671cb878e68647ac835854cdd7459cce_SeqId_44_ 2023-02-11 03:54:44,620 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.HStore(849): Loaded HFile hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__d13c0ds831vdl4tqoa44hkr607cmcvn47ieojlrniipli0iob85gk7lkvjtnkbq2/family1/2cdc03c039b34093ae25f40bb029a290 into 7e59b0be260b97681ec2d05527ed1bde/family1 as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/7e59b0be260b97681ec2d05527ed1bde/family1/671cb878e68647ac835854cdd7459cce_SeqId_44_ - updating store file list. 2023-02-11 03:54:44,623 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.HStore(882): Loaded HFile hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/7e59b0be260b97681ec2d05527ed1bde/family1/671cb878e68647ac835854cdd7459cce_SeqId_44_ into 7e59b0be260b97681ec2d05527ed1bde/family1 2023-02-11 03:54:44,623 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.HStore(855): Successfully loaded hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__d13c0ds831vdl4tqoa44hkr607cmcvn47ieojlrniipli0iob85gk7lkvjtnkbq2/family1/2cdc03c039b34093ae25f40bb029a290 into 7e59b0be260b97681ec2d05527ed1bde/family1 (new location: hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/7e59b0be260b97681ec2d05527ed1bde/family1/671cb878e68647ac835854cdd7459cce_SeqId_44_) 2023-02-11 03:54:44,623 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__d13c0ds831vdl4tqoa44hkr607cmcvn47ieojlrniipli0iob85gk7lkvjtnkbq2/family1/2cdc03c039b34093ae25f40bb029a290 2023-02-11 03:54:44,623 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] compactions.SortedCompactionPolicy(75): Selecting compaction from 11 store files, 0 compacting, 11 eligible, 16 blocking 2023-02-11 03:54:44,626 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 10 files of size 51565 starting at candidate #1 after considering 44 permutations with 44 in ratio 2023-02-11 03:54:44,626 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.HStore(1912): 7e59b0be260b97681ec2d05527ed1bde/family1 is initiating minor compaction 2023-02-11 03:54:44,626 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@79daa4f2; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-11 03:54:44,626 INFO [RS:0;jenkins-hbase4:37011-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 7e59b0be260b97681ec2d05527ed1bde/family1 in testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676087672040.7e59b0be260b97681ec2d05527ed1bde. 2023-02-11 03:54:44,626 INFO [RS:0;jenkins-hbase4:37011-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/7e59b0be260b97681ec2d05527ed1bde/family1/6ef47ae302584c6c900b22a4e2ff5cff, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/7e59b0be260b97681ec2d05527ed1bde/family1/08ac5f1d679041beb01fc7926f1c1233, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/7e59b0be260b97681ec2d05527ed1bde/family1/94e9c6c9026042d386f284b3000d27bd, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/7e59b0be260b97681ec2d05527ed1bde/family1/e1c919793ac34ed08250216d47286715, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/7e59b0be260b97681ec2d05527ed1bde/family1/19bc8225da374841a0584d285026c93b, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/7e59b0be260b97681ec2d05527ed1bde/family1/bfbba689fa2d4c129ad1ca9e24762cd2, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/7e59b0be260b97681ec2d05527ed1bde/family1/8203077f03534604846eb7a86101fc21, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/7e59b0be260b97681ec2d05527ed1bde/family1/f7e01afc549f44c9b55237ba2f97fe5d, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/7e59b0be260b97681ec2d05527ed1bde/family1/f68c7a9e88ed43839ce25eeee70560d6, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/7e59b0be260b97681ec2d05527ed1bde/family1/671cb878e68647ac835854cdd7459cce_SeqId_44_] into tmpdir=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/7e59b0be260b97681ec2d05527ed1bde/.tmp, totalSize=50.4 K 2023-02-11 03:54:44,626 DEBUG [RS:0;jenkins-hbase4:37011-shortCompactions-0] compactions.Compactor(207): Compacting 6ef47ae302584c6c900b22a4e2ff5cff, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=9 2023-02-11 03:54:44,626 DEBUG [RS:0;jenkins-hbase4:37011-shortCompactions-0] compactions.Compactor(207): Compacting 08ac5f1d679041beb01fc7926f1c1233, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=13 2023-02-11 03:54:44,627 DEBUG [RS:0;jenkins-hbase4:37011-shortCompactions-0] compactions.Compactor(207): Compacting 94e9c6c9026042d386f284b3000d27bd, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=17 2023-02-11 03:54:44,627 DEBUG [RS:0;jenkins-hbase4:37011-shortCompactions-0] compactions.Compactor(207): Compacting e1c919793ac34ed08250216d47286715, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=21 2023-02-11 03:54:44,627 DEBUG [RS:0;jenkins-hbase4:37011-shortCompactions-0] compactions.Compactor(207): Compacting 19bc8225da374841a0584d285026c93b, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=25 2023-02-11 03:54:44,627 DEBUG [RS:0;jenkins-hbase4:37011-shortCompactions-0] compactions.Compactor(207): Compacting bfbba689fa2d4c129ad1ca9e24762cd2, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=29 2023-02-11 03:54:44,627 DEBUG [RS:0;jenkins-hbase4:37011-shortCompactions-0] compactions.Compactor(207): Compacting 8203077f03534604846eb7a86101fc21, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33 2023-02-11 03:54:44,627 DEBUG [RS:0;jenkins-hbase4:37011-shortCompactions-0] compactions.Compactor(207): Compacting f7e01afc549f44c9b55237ba2f97fe5d, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=37 2023-02-11 03:54:44,627 DEBUG [RS:0;jenkins-hbase4:37011-shortCompactions-0] compactions.Compactor(207): Compacting f68c7a9e88ed43839ce25eeee70560d6, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=41 2023-02-11 03:54:44,627 DEBUG [RS:0;jenkins-hbase4:37011-shortCompactions-0] compactions.Compactor(207): Compacting 671cb878e68647ac835854cdd7459cce_SeqId_44_, keycount=4, bloomtype=ROW, size=5.6 K, encoding=NONE, compression=NONE, seqNum=45 2023-02-11 03:54:44,628 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-02-11 03:54:44,629 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-11 03:54:44,630 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151228da0124039cda4f05a1b53d0b6deb305c to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151228da0124039cda4f05a1b53d0b6deb305c 2023-02-11 03:54:44,632 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151231a6db5f18b2ea4f059ea21ba788abe5aa to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151231a6db5f18b2ea4f059ea21ba788abe5aa 2023-02-11 03:54:44,633 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20160101d5a8a5a3318041d7bede4114aec8652a to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20160101d5a8a5a3318041d7bede4114aec8652a 2023-02-11 03:54:44,634 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201601038dbf50b4a40d453ba2a8261a2e5741af to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201601038dbf50b4a40d453ba2a8261a2e5741af 2023-02-11 03:54:44,634 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 4 expired mob files are deleted 2023-02-11 03:54:44,634 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151228' 2023-02-11 03:54:44,634 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-11 03:54:44,635 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-11 03:54:44,635 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-11 03:54:44,635 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-11 03:54:44,635 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-11 03:54:44,635 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-11 03:54:44,635 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-11 03:54:44,635 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-11 03:54:44,659 INFO [RS:0;jenkins-hbase4:37011-shortCompactions-0] throttle.PressureAwareThroughputController(145): 7e59b0be260b97681ec2d05527ed1bde#family1#compaction#60 average throughput is 1.59 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 61.54 MB/second 2023-02-11 03:54:45,479 DEBUG [RS:0;jenkins-hbase4:37011-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/7e59b0be260b97681ec2d05527ed1bde/.tmp/family1/37941af706ef42d78573e280d34c0e0d as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/7e59b0be260b97681ec2d05527ed1bde/family1/37941af706ef42d78573e280d34c0e0d 2023-02-11 03:54:45,483 INFO [RS:0;jenkins-hbase4:37011-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 10 file(s) in 7e59b0be260b97681ec2d05527ed1bde/family1 of 7e59b0be260b97681ec2d05527ed1bde into 37941af706ef42d78573e280d34c0e0d(size=6.8 K), total size for store is 198.2 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-11 03:54:45,483 DEBUG [RS:0;jenkins-hbase4:37011-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 7e59b0be260b97681ec2d05527ed1bde: 2023-02-11 03:54:45,484 INFO [RS:0;jenkins-hbase4:37011-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676087672040.7e59b0be260b97681ec2d05527ed1bde., storeName=family1, fileCount=10, fileSize=50.4 K (5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.6 K), priority=2, time=1676087684626; duration=0sec 2023-02-11 03:54:45,484 DEBUG [RS:0;jenkins-hbase4:37011-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-11 03:54:45,498 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/7e59b0be260b97681ec2d05527ed1bde/family1/6ef47ae302584c6c900b22a4e2ff5cff, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/7e59b0be260b97681ec2d05527ed1bde/family1/08ac5f1d679041beb01fc7926f1c1233, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/7e59b0be260b97681ec2d05527ed1bde/family1/94e9c6c9026042d386f284b3000d27bd, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/7e59b0be260b97681ec2d05527ed1bde/family1/e1c919793ac34ed08250216d47286715, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/7e59b0be260b97681ec2d05527ed1bde/family1/19bc8225da374841a0584d285026c93b, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/7e59b0be260b97681ec2d05527ed1bde/family1/bfbba689fa2d4c129ad1ca9e24762cd2, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/7e59b0be260b97681ec2d05527ed1bde/family1/8203077f03534604846eb7a86101fc21, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/7e59b0be260b97681ec2d05527ed1bde/family1/f7e01afc549f44c9b55237ba2f97fe5d, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/7e59b0be260b97681ec2d05527ed1bde/family1/f68c7a9e88ed43839ce25eeee70560d6, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/7e59b0be260b97681ec2d05527ed1bde/family1/671cb878e68647ac835854cdd7459cce_SeqId_44_] to archive 2023-02-11 03:54:45,498 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015120574a3eb316e0e484d8fff44a7fd489cb5 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e2015120574a3eb316e0e484d8fff44a7fd489cb5 2023-02-11 03:54:45,499 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-11 03:54:45,499 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-02-11 03:54:45,500 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/7e59b0be260b97681ec2d05527ed1bde/family1/6ef47ae302584c6c900b22a4e2ff5cff to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/7e59b0be260b97681ec2d05527ed1bde/family1/6ef47ae302584c6c900b22a4e2ff5cff 2023-02-11 03:54:45,501 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/7e59b0be260b97681ec2d05527ed1bde/family1/08ac5f1d679041beb01fc7926f1c1233 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/7e59b0be260b97681ec2d05527ed1bde/family1/08ac5f1d679041beb01fc7926f1c1233 2023-02-11 03:54:45,502 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/7e59b0be260b97681ec2d05527ed1bde/family1/94e9c6c9026042d386f284b3000d27bd to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/7e59b0be260b97681ec2d05527ed1bde/family1/94e9c6c9026042d386f284b3000d27bd 2023-02-11 03:54:45,503 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/7e59b0be260b97681ec2d05527ed1bde/family1/e1c919793ac34ed08250216d47286715 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/7e59b0be260b97681ec2d05527ed1bde/family1/e1c919793ac34ed08250216d47286715 2023-02-11 03:54:45,504 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/7e59b0be260b97681ec2d05527ed1bde/family1/19bc8225da374841a0584d285026c93b to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/7e59b0be260b97681ec2d05527ed1bde/family1/19bc8225da374841a0584d285026c93b 2023-02-11 03:54:45,505 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/7e59b0be260b97681ec2d05527ed1bde/family1/bfbba689fa2d4c129ad1ca9e24762cd2 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/7e59b0be260b97681ec2d05527ed1bde/family1/bfbba689fa2d4c129ad1ca9e24762cd2 2023-02-11 03:54:45,506 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/7e59b0be260b97681ec2d05527ed1bde/family1/8203077f03534604846eb7a86101fc21 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/7e59b0be260b97681ec2d05527ed1bde/family1/8203077f03534604846eb7a86101fc21 2023-02-11 03:54:45,507 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/7e59b0be260b97681ec2d05527ed1bde/family1/f7e01afc549f44c9b55237ba2f97fe5d to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/7e59b0be260b97681ec2d05527ed1bde/family1/f7e01afc549f44c9b55237ba2f97fe5d 2023-02-11 03:54:45,508 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/7e59b0be260b97681ec2d05527ed1bde/family1/f68c7a9e88ed43839ce25eeee70560d6 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/7e59b0be260b97681ec2d05527ed1bde/family1/f68c7a9e88ed43839ce25eeee70560d6 2023-02-11 03:54:45,509 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/7e59b0be260b97681ec2d05527ed1bde/family1/671cb878e68647ac835854cdd7459cce_SeqId_44_ to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/7e59b0be260b97681ec2d05527ed1bde/family1/671cb878e68647ac835854cdd7459cce_SeqId_44_ 2023-02-11 03:54:45,522 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151130/family1/069d12f823404583aa9decc547c51b7d first=Optional[r00] last=Optional[r2] 2023-02-11 03:54:45,523 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676087672040.7e59b0be260b97681ec2d05527ed1bde., hostname=jenkins-hbase4.apache.org,37011,1676087618428, seqNum=2 for row with hfile group [{family1,hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151130/family1/069d12f823404583aa9decc547c51b7d}] 2023-02-11 03:54:45,524 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.HStore(763): Validating hfile at hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151130/family1/069d12f823404583aa9decc547c51b7d for inclusion in 7e59b0be260b97681ec2d05527ed1bde/family1 2023-02-11 03:54:45,527 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.HStore(775): HFile bounds: first=r00 last=r2 2023-02-11 03:54:45,527 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.HStore(777): Region bounds: first= last= 2023-02-11 03:54:45,527 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.HRegion(2445): Flush status journal for 7e59b0be260b97681ec2d05527ed1bde: 2023-02-11 03:54:45,528 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151130/family1/069d12f823404583aa9decc547c51b7d to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__5iopd5qv8slhtapssp5m9d4pa1s0c8fhg40ldkb206g89r0gh1l117dngmqtlcr9/family1/069d12f823404583aa9decc547c51b7d 2023-02-11 03:54:45,530 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__5iopd5qv8slhtapssp5m9d4pa1s0c8fhg40ldkb206g89r0gh1l117dngmqtlcr9/family1/069d12f823404583aa9decc547c51b7d as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/7e59b0be260b97681ec2d05527ed1bde/family1/fd2eae93d7734b649f2cc8a55251170d_SeqId_47_ 2023-02-11 03:54:45,530 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.HStore(849): Loaded HFile hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__5iopd5qv8slhtapssp5m9d4pa1s0c8fhg40ldkb206g89r0gh1l117dngmqtlcr9/family1/069d12f823404583aa9decc547c51b7d into 7e59b0be260b97681ec2d05527ed1bde/family1 as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/7e59b0be260b97681ec2d05527ed1bde/family1/fd2eae93d7734b649f2cc8a55251170d_SeqId_47_ - updating store file list. 2023-02-11 03:54:45,534 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.HStore(882): Loaded HFile hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/7e59b0be260b97681ec2d05527ed1bde/family1/fd2eae93d7734b649f2cc8a55251170d_SeqId_47_ into 7e59b0be260b97681ec2d05527ed1bde/family1 2023-02-11 03:54:45,534 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.HStore(855): Successfully loaded hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__5iopd5qv8slhtapssp5m9d4pa1s0c8fhg40ldkb206g89r0gh1l117dngmqtlcr9/family1/069d12f823404583aa9decc547c51b7d into 7e59b0be260b97681ec2d05527ed1bde/family1 (new location: hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/7e59b0be260b97681ec2d05527ed1bde/family1/fd2eae93d7734b649f2cc8a55251170d_SeqId_47_) 2023-02-11 03:54:45,534 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__5iopd5qv8slhtapssp5m9d4pa1s0c8fhg40ldkb206g89r0gh1l117dngmqtlcr9/family1/069d12f823404583aa9decc547c51b7d 2023-02-11 03:54:45,534 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-02-11 03:54:45,535 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 399433 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-02-11 03:54:45,535 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.HStore(1912): 7e59b0be260b97681ec2d05527ed1bde/family1 is initiating minor compaction (all files) 2023-02-11 03:54:45,535 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@60faabdc; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-11 03:54:45,535 INFO [RS:0;jenkins-hbase4:37011-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 7e59b0be260b97681ec2d05527ed1bde/family1 in testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676087672040.7e59b0be260b97681ec2d05527ed1bde. 2023-02-11 03:54:45,536 INFO [RS:0;jenkins-hbase4:37011-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/7e59b0be260b97681ec2d05527ed1bde/family1/c4e5a19573734e029b5e6a9bc078519a, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/7e59b0be260b97681ec2d05527ed1bde/family1/37941af706ef42d78573e280d34c0e0d, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/7e59b0be260b97681ec2d05527ed1bde/family1/fd2eae93d7734b649f2cc8a55251170d_SeqId_47_] into tmpdir=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/7e59b0be260b97681ec2d05527ed1bde/.tmp, totalSize=390.1 K 2023-02-11 03:54:45,536 DEBUG [RS:0;jenkins-hbase4:37011-shortCompactions-0] compactions.Compactor(207): Compacting c4e5a19573734e029b5e6a9bc078519a, keycount=1000, bloomtype=ROW, size=191.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1448878842014 2023-02-11 03:54:45,536 DEBUG [RS:0;jenkins-hbase4:37011-shortCompactions-0] compactions.Compactor(207): Compacting 37941af706ef42d78573e280d34c0e0d, keycount=9, bloomtype=ROW, size=6.8 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1441621242014 2023-02-11 03:54:45,537 DEBUG [RS:0;jenkins-hbase4:37011-shortCompactions-0] compactions.Compactor(207): Compacting fd2eae93d7734b649f2cc8a55251170d_SeqId_47_, keycount=1002, bloomtype=ROW, size=191.8 K, encoding=NONE, compression=NONE, seqNum=48, earliestPutTs=1448878842014 2023-02-11 03:54:45,537 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-02-11 03:54:45,538 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-11 03:54:45,539 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151130eb8a06e8ba7f407db81ec47939706106 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151130eb8a06e8ba7f407db81ec47939706106 2023-02-11 03:54:45,540 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201512013c311260cdaa4a179c410de6ce59854e to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201512013c311260cdaa4a179c410de6ce59854e 2023-02-11 03:54:45,541 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151205eb8f1ddd83834018a712ccf29db5da76 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151205eb8f1ddd83834018a712ccf29db5da76 2023-02-11 03:54:45,541 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 3 expired mob files are deleted 2023-02-11 03:54:45,541 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151130' 2023-02-11 03:54:45,542 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 3 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-11 03:54:45,542 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676087621935] compactions.PartitionedMobCompactor(354): After compaction, there are 5 mob files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-02-11 03:54:45,542 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676087621935] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-11 03:54:45,543 DEBUG [PEWorker-4] locking.LockProcedure(242): UNLOCKED pid=42, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-02-11 03:54:45,545 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=42, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE in 1.8100 sec 2023-02-11 03:54:45,555 INFO [RS:0;jenkins-hbase4:37011-shortCompactions-0] throttle.PressureAwareThroughputController(145): 7e59b0be260b97681ec2d05527ed1bde#family1#compaction#61 average throughput is 20.07 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 61.54 MB/second 2023-02-11 03:54:45,617 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] master.HMaster$12(2441): Client=jenkins//172.31.14.131 modify table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly from 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} to 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'} 2023-02-11 03:54:45,618 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] procedure2.ProcedureExecutor(1029): Stored pid=43, state=RUNNABLE:MODIFY_TABLE_PREPARE; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:46,026 DEBUG [PEWorker-5] util.FSTableDescriptors(570): Wrote into hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/.tabledesc/.tableinfo.0000000002 2023-02-11 03:54:46,027 DEBUG [PEWorker-5] util.FSTableDescriptors(519): Deleted hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/.tabledesc/.tableinfo.0000000001 2023-02-11 03:54:46,027 INFO [PEWorker-5] util.FSTableDescriptors(293): Updated tableinfo=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/.tabledesc/.tableinfo.0000000002 2023-02-11 03:54:46,029 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=44, ppid=43, state=RUNNABLE:REOPEN_TABLE_REGIONS_GET_REGIONS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly}] 2023-02-11 03:54:46,031 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=7e59b0be260b97681ec2d05527ed1bde, REOPEN/MOVE}] 2023-02-11 03:54:46,032 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=7e59b0be260b97681ec2d05527ed1bde, REOPEN/MOVE 2023-02-11 03:54:46,033 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=7e59b0be260b97681ec2d05527ed1bde, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:46,033 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676087672040.7e59b0be260b97681ec2d05527ed1bde.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1676087686033"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676087686033"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676087686033"}]},"ts":"1676087686033"} 2023-02-11 03:54:46,034 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=46, ppid=45, state=RUNNABLE; CloseRegionProcedure 7e59b0be260b97681ec2d05527ed1bde, server=jenkins-hbase4.apache.org,37011,1676087618428}] 2023-02-11 03:54:46,186 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 7e59b0be260b97681ec2d05527ed1bde 2023-02-11 03:54:46,186 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 7e59b0be260b97681ec2d05527ed1bde, disabling compactions & flushes 2023-02-11 03:54:46,186 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1857): waiting for 1 compactions to complete for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676087672040.7e59b0be260b97681ec2d05527ed1bde. 2023-02-11 03:54:46,776 DEBUG [RS:0;jenkins-hbase4:37011-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/7e59b0be260b97681ec2d05527ed1bde/.tmp/family1/fc95456c53b24e878864598023112023 as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/7e59b0be260b97681ec2d05527ed1bde/family1/fc95456c53b24e878864598023112023 2023-02-11 03:54:46,780 INFO [RS:0;jenkins-hbase4:37011-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in 7e59b0be260b97681ec2d05527ed1bde/family1 of 7e59b0be260b97681ec2d05527ed1bde into fc95456c53b24e878864598023112023(size=193.0 K), total size for store is 193.0 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-02-11 03:54:46,780 DEBUG [RS:0;jenkins-hbase4:37011-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 7e59b0be260b97681ec2d05527ed1bde: 2023-02-11 03:54:46,780 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676087672040.7e59b0be260b97681ec2d05527ed1bde. 2023-02-11 03:54:46,780 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676087672040.7e59b0be260b97681ec2d05527ed1bde. 2023-02-11 03:54:46,780 INFO [RS:0;jenkins-hbase4:37011-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676087672040.7e59b0be260b97681ec2d05527ed1bde., storeName=family1, fileCount=3, fileSize=390.1 K (191.4 K, 6.8 K, 191.8 K), priority=2, time=1676087685535; duration=1sec 2023-02-11 03:54:46,780 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676087672040.7e59b0be260b97681ec2d05527ed1bde. after waiting 0 ms 2023-02-11 03:54:46,780 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676087672040.7e59b0be260b97681ec2d05527ed1bde. 2023-02-11 03:54:46,780 DEBUG [RS:0;jenkins-hbase4:37011-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-11 03:54:46,781 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676087672040.7e59b0be260b97681ec2d05527ed1bde.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/7e59b0be260b97681ec2d05527ed1bde/family1/c4e5a19573734e029b5e6a9bc078519a, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/7e59b0be260b97681ec2d05527ed1bde/family1/37941af706ef42d78573e280d34c0e0d, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/7e59b0be260b97681ec2d05527ed1bde/family1/fd2eae93d7734b649f2cc8a55251170d_SeqId_47_] to archive 2023-02-11 03:54:46,782 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676087672040.7e59b0be260b97681ec2d05527ed1bde.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-11 03:54:46,783 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676087672040.7e59b0be260b97681ec2d05527ed1bde.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/7e59b0be260b97681ec2d05527ed1bde/family1/c4e5a19573734e029b5e6a9bc078519a to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/7e59b0be260b97681ec2d05527ed1bde/family1/c4e5a19573734e029b5e6a9bc078519a 2023-02-11 03:54:46,784 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676087672040.7e59b0be260b97681ec2d05527ed1bde.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/7e59b0be260b97681ec2d05527ed1bde/family1/37941af706ef42d78573e280d34c0e0d to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/7e59b0be260b97681ec2d05527ed1bde/family1/37941af706ef42d78573e280d34c0e0d 2023-02-11 03:54:46,785 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676087672040.7e59b0be260b97681ec2d05527ed1bde.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/7e59b0be260b97681ec2d05527ed1bde/family1/fd2eae93d7734b649f2cc8a55251170d_SeqId_47_ to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/7e59b0be260b97681ec2d05527ed1bde/family1/fd2eae93d7734b649f2cc8a55251170d_SeqId_47_ 2023-02-11 03:54:46,790 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/7e59b0be260b97681ec2d05527ed1bde/recovered.edits/50.seqid, newMaxSeqId=50, maxSeqId=1 2023-02-11 03:54:46,790 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676087672040.7e59b0be260b97681ec2d05527ed1bde. 2023-02-11 03:54:46,791 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 7e59b0be260b97681ec2d05527ed1bde: Waiting for close lock at 1676087686186Running coprocessor pre-close hooks at 1676087686186Disabling compacts and flushes for region at 1676087686186Disabling writes for close at 1676087686780 (+594 ms)Writing region close event to WAL at 1676087686787 (+7 ms)Running coprocessor post-close hooks at 1676087686790 (+3 ms)Closed at 1676087686790 2023-02-11 03:54:46,791 WARN [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(3509): Not adding moved region record: 7e59b0be260b97681ec2d05527ed1bde to self. 2023-02-11 03:54:46,792 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 7e59b0be260b97681ec2d05527ed1bde 2023-02-11 03:54:46,792 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=7e59b0be260b97681ec2d05527ed1bde, regionState=CLOSED 2023-02-11 03:54:46,792 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676087672040.7e59b0be260b97681ec2d05527ed1bde.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1676087686792"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676087686792"}]},"ts":"1676087686792"} 2023-02-11 03:54:46,795 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=46, resume processing ppid=45 2023-02-11 03:54:46,795 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 7e59b0be260b97681ec2d05527ed1bde, server=jenkins-hbase4.apache.org,37011,1676087618428 in 760 msec 2023-02-11 03:54:46,795 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=7e59b0be260b97681ec2d05527ed1bde, REOPEN/MOVE; state=CLOSED, location=jenkins-hbase4.apache.org,37011,1676087618428; forceNewPlan=false, retain=true 2023-02-11 03:54:46,946 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=7e59b0be260b97681ec2d05527ed1bde, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:46,946 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676087672040.7e59b0be260b97681ec2d05527ed1bde.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1676087686946"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676087686946"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676087686946"}]},"ts":"1676087686946"} 2023-02-11 03:54:46,947 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=47, ppid=45, state=RUNNABLE; OpenRegionProcedure 7e59b0be260b97681ec2d05527ed1bde, server=jenkins-hbase4.apache.org,37011,1676087618428}] 2023-02-11 03:54:47,102 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676087672040.7e59b0be260b97681ec2d05527ed1bde. 2023-02-11 03:54:47,102 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 7e59b0be260b97681ec2d05527ed1bde, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676087672040.7e59b0be260b97681ec2d05527ed1bde.', STARTKEY => '', ENDKEY => ''} 2023-02-11 03:54:47,102 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 7e59b0be260b97681ec2d05527ed1bde 2023-02-11 03:54:47,102 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676087672040.7e59b0be260b97681ec2d05527ed1bde.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-11 03:54:47,102 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 7e59b0be260b97681ec2d05527ed1bde 2023-02-11 03:54:47,102 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 7e59b0be260b97681ec2d05527ed1bde 2023-02-11 03:54:47,103 INFO [StoreOpener-7e59b0be260b97681ec2d05527ed1bde-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 7e59b0be260b97681ec2d05527ed1bde 2023-02-11 03:54:47,104 DEBUG [StoreOpener-7e59b0be260b97681ec2d05527ed1bde-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/7e59b0be260b97681ec2d05527ed1bde/family1 2023-02-11 03:54:47,104 DEBUG [StoreOpener-7e59b0be260b97681ec2d05527ed1bde-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/7e59b0be260b97681ec2d05527ed1bde/family1 2023-02-11 03:54:47,105 INFO [StoreOpener-7e59b0be260b97681ec2d05527ed1bde-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 7e59b0be260b97681ec2d05527ed1bde columnFamilyName family1 2023-02-11 03:54:47,109 DEBUG [StoreOpener-7e59b0be260b97681ec2d05527ed1bde-1] regionserver.HStore(539): loaded hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/7e59b0be260b97681ec2d05527ed1bde/family1/fc95456c53b24e878864598023112023 2023-02-11 03:54:47,109 INFO [StoreOpener-7e59b0be260b97681ec2d05527ed1bde-1] regionserver.HStore(310): Store=7e59b0be260b97681ec2d05527ed1bde/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 03:54:47,110 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/7e59b0be260b97681ec2d05527ed1bde 2023-02-11 03:54:47,111 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/7e59b0be260b97681ec2d05527ed1bde 2023-02-11 03:54:47,113 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 7e59b0be260b97681ec2d05527ed1bde 2023-02-11 03:54:47,114 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 7e59b0be260b97681ec2d05527ed1bde; next sequenceid=51; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=71369349, jitterRate=0.06348617374897003}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-11 03:54:47,114 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 7e59b0be260b97681ec2d05527ed1bde: Running coprocessor pre-open hook at 1676087687102Writing region info on filesystem at 1676087687102Initializing all the Stores at 1676087687103 (+1 ms)Instantiating store for column family {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'} at 1676087687103Cleaning up temporary data from old regions at 1676087687111 (+8 ms)Cleaning up detritus from prior splits at 1676087687112 (+1 ms)Running coprocessor post-open hooks at 1676087687114 (+2 ms)Region opened successfully at 1676087687114 2023-02-11 03:54:47,114 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676087672040.7e59b0be260b97681ec2d05527ed1bde., pid=47, masterSystemTime=1676087687099 2023-02-11 03:54:47,116 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676087672040.7e59b0be260b97681ec2d05527ed1bde. 2023-02-11 03:54:47,116 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676087672040.7e59b0be260b97681ec2d05527ed1bde. 2023-02-11 03:54:47,116 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=7e59b0be260b97681ec2d05527ed1bde, regionState=OPEN, openSeqNum=51, regionLocation=jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:47,117 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676087672040.7e59b0be260b97681ec2d05527ed1bde.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1676087687116"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676087687116"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676087687116"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676087687116"}]},"ts":"1676087687116"} 2023-02-11 03:54:47,119 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=47, resume processing ppid=45 2023-02-11 03:54:47,119 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=47, ppid=45, state=SUCCESS; OpenRegionProcedure 7e59b0be260b97681ec2d05527ed1bde, server=jenkins-hbase4.apache.org,37011,1676087618428 in 171 msec 2023-02-11 03:54:47,121 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=45, resume processing ppid=44 2023-02-11 03:54:47,121 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=7e59b0be260b97681ec2d05527ed1bde, REOPEN/MOVE in 1.0880 sec 2023-02-11 03:54:47,122 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=44, resume processing ppid=43 2023-02-11 03:54:47,122 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=44, ppid=43, state=SUCCESS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly in 1.0920 sec 2023-02-11 03:54:47,124 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=43, state=SUCCESS; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly in 1.5060 sec 2023-02-11 03:54:47,124 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] master.MasterRpcServices(1227): Checking to see if procedure is done pid=43 2023-02-11 03:54:47,125 INFO [Listener at localhost/46205] compactions.TestMobCompactor(278): alter status finished 2023-02-11 03:54:47,126 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=39921] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'}] of the table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:47,127 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676087621935] procedure2.ProcedureExecutor(1029): Stored pid=48, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-02-11 03:54:47,128 DEBUG [PEWorker-1] locking.LockProcedure(309): LOCKED pid=48, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-02-11 03:54:47,128 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676087621935] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-11 03:54:47,128 INFO [PEWorker-1] procedure2.TimeoutExecutorThread(81): ADDED pid=48, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1676088287128 2023-02-11 03:54:47,129 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676087621935] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-02-11 03:54:47,130 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676087621935] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 5 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column 'family1' 2023-02-11 03:54:47,130 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676087621935] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-02-11 03:54:47,130 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676087621935] zookeeper.ReadOnlyZKClient(139): Connect 0x38bb69c6 to 127.0.0.1:53383 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-11 03:54:47,146 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676087621935] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@213a2f46, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-11 03:54:47,147 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-11 03:54:47,147 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-11 03:54:47,147 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-11 03:54:47,147 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-11 03:54:47,147 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-11 03:54:47,972 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015112830d98ee7a963436b983efd0acab470ed to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e2015112830d98ee7a963436b983efd0acab470ed 2023-02-11 03:54:47,973 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-02-11 03:54:47,985 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-11 03:54:47,987 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:44474, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-11 03:54:47,993 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-02-11 03:54:47,995 INFO [RS-EventLoopGroup-1-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:54216, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-02-11 03:54:48,003 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151101/family1/dafe5fcfb3564aa8a677a7948233e873 first=Optional[r05] last=Optional[r8] 2023-02-11 03:54:48,003 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676087672040.7e59b0be260b97681ec2d05527ed1bde., hostname=jenkins-hbase4.apache.org,37011,1676087618428, seqNum=51 for row with hfile group [{family1,hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151101/family1/dafe5fcfb3564aa8a677a7948233e873}] 2023-02-11 03:54:48,008 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.HStore(763): Validating hfile at hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151101/family1/dafe5fcfb3564aa8a677a7948233e873 for inclusion in 7e59b0be260b97681ec2d05527ed1bde/family1 2023-02-11 03:54:48,010 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.HStore(775): HFile bounds: first=r05 last=r8 2023-02-11 03:54:48,010 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.HStore(777): Region bounds: first= last= 2023-02-11 03:54:48,010 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.HRegion(2445): Flush status journal for 7e59b0be260b97681ec2d05527ed1bde: 2023-02-11 03:54:48,011 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151101/family1/dafe5fcfb3564aa8a677a7948233e873 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__l52ao4bjphass6kimf2tukjgkooi0frhoh6b5js32tamoda49a84jplmq6bv7tum/family1/dafe5fcfb3564aa8a677a7948233e873 2023-02-11 03:54:48,014 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__l52ao4bjphass6kimf2tukjgkooi0frhoh6b5js32tamoda49a84jplmq6bv7tum/family1/dafe5fcfb3564aa8a677a7948233e873 as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/7e59b0be260b97681ec2d05527ed1bde/family1/77f4e1f915eb4dfebae97d0ac7c88d04_SeqId_53_ 2023-02-11 03:54:48,015 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.HStore(849): Loaded HFile hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__l52ao4bjphass6kimf2tukjgkooi0frhoh6b5js32tamoda49a84jplmq6bv7tum/family1/dafe5fcfb3564aa8a677a7948233e873 into 7e59b0be260b97681ec2d05527ed1bde/family1 as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/7e59b0be260b97681ec2d05527ed1bde/family1/77f4e1f915eb4dfebae97d0ac7c88d04_SeqId_53_ - updating store file list. 2023-02-11 03:54:48,018 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.HStore(882): Loaded HFile hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/7e59b0be260b97681ec2d05527ed1bde/family1/77f4e1f915eb4dfebae97d0ac7c88d04_SeqId_53_ into 7e59b0be260b97681ec2d05527ed1bde/family1 2023-02-11 03:54:48,018 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.HStore(855): Successfully loaded hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__l52ao4bjphass6kimf2tukjgkooi0frhoh6b5js32tamoda49a84jplmq6bv7tum/family1/dafe5fcfb3564aa8a677a7948233e873 into 7e59b0be260b97681ec2d05527ed1bde/family1 (new location: hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/7e59b0be260b97681ec2d05527ed1bde/family1/77f4e1f915eb4dfebae97d0ac7c88d04_SeqId_53_) 2023-02-11 03:54:48,019 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__l52ao4bjphass6kimf2tukjgkooi0frhoh6b5js32tamoda49a84jplmq6bv7tum/family1/dafe5fcfb3564aa8a677a7948233e873 2023-02-11 03:54:48,020 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-02-11 03:54:48,021 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-11 03:54:48,022 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201511206d60b9c2ce0b4ff7b5b20982466710f6 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201511206d60b9c2ce0b4ff7b5b20982466710f6 2023-02-11 03:54:48,023 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151128e3a6f16ee6d04ef390cfc46c667b7d5d to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151128e3a6f16ee6d04ef390cfc46c667b7d5d 2023-02-11 03:54:48,023 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2 expired mob files are deleted 2023-02-11 03:54:48,023 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151101' 2023-02-11 03:54:48,023 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 2 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-11 03:54:48,024 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-11 03:54:48,024 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-11 03:54:48,024 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676087621935] compactions.PartitionedMobCompactor(354): After compaction, there are 4 mob files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-02-11 03:54:48,024 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676087621935] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-11 03:54:48,024 DEBUG [PEWorker-3] locking.LockProcedure(242): UNLOCKED pid=48, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-02-11 03:54:48,026 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=48, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE in 897 msec 2023-02-11 03:54:48,077 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] master.HMaster$12(2441): Client=jenkins//172.31.14.131 modify table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly from 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'} to 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} 2023-02-11 03:54:48,078 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] procedure2.ProcedureExecutor(1029): Stored pid=49, state=RUNNABLE:MODIFY_TABLE_PREPARE; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:48,487 DEBUG [PEWorker-4] util.FSTableDescriptors(570): Wrote into hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/.tabledesc/.tableinfo.0000000003 2023-02-11 03:54:48,488 DEBUG [PEWorker-4] util.FSTableDescriptors(519): Deleted hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/.tabledesc/.tableinfo.0000000002 2023-02-11 03:54:48,488 INFO [PEWorker-4] util.FSTableDescriptors(293): Updated tableinfo=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/.tabledesc/.tableinfo.0000000003 2023-02-11 03:54:48,491 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=50, ppid=49, state=RUNNABLE:REOPEN_TABLE_REGIONS_GET_REGIONS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly}] 2023-02-11 03:54:48,493 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=51, ppid=50, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=7e59b0be260b97681ec2d05527ed1bde, REOPEN/MOVE}] 2023-02-11 03:54:48,493 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=51, ppid=50, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=7e59b0be260b97681ec2d05527ed1bde, REOPEN/MOVE 2023-02-11 03:54:48,494 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=51 updating hbase:meta row=7e59b0be260b97681ec2d05527ed1bde, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:48,494 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676087672040.7e59b0be260b97681ec2d05527ed1bde.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1676087688494"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676087688494"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676087688494"}]},"ts":"1676087688494"} 2023-02-11 03:54:48,495 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=52, ppid=51, state=RUNNABLE; CloseRegionProcedure 7e59b0be260b97681ec2d05527ed1bde, server=jenkins-hbase4.apache.org,37011,1676087618428}] 2023-02-11 03:54:48,647 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 7e59b0be260b97681ec2d05527ed1bde 2023-02-11 03:54:48,647 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 7e59b0be260b97681ec2d05527ed1bde, disabling compactions & flushes 2023-02-11 03:54:48,647 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676087672040.7e59b0be260b97681ec2d05527ed1bde. 2023-02-11 03:54:48,647 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676087672040.7e59b0be260b97681ec2d05527ed1bde. 2023-02-11 03:54:48,647 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676087672040.7e59b0be260b97681ec2d05527ed1bde. after waiting 0 ms 2023-02-11 03:54:48,647 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676087672040.7e59b0be260b97681ec2d05527ed1bde. 2023-02-11 03:54:48,653 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/7e59b0be260b97681ec2d05527ed1bde/recovered.edits/55.seqid, newMaxSeqId=55, maxSeqId=50 2023-02-11 03:54:48,653 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676087672040.7e59b0be260b97681ec2d05527ed1bde. 2023-02-11 03:54:48,654 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 7e59b0be260b97681ec2d05527ed1bde: Waiting for close lock at 1676087688647Running coprocessor pre-close hooks at 1676087688647Disabling compacts and flushes for region at 1676087688647Disabling writes for close at 1676087688647Writing region close event to WAL at 1676087688650 (+3 ms)Running coprocessor post-close hooks at 1676087688653 (+3 ms)Closed at 1676087688653 2023-02-11 03:54:48,654 WARN [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(3509): Not adding moved region record: 7e59b0be260b97681ec2d05527ed1bde to self. 2023-02-11 03:54:48,655 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 7e59b0be260b97681ec2d05527ed1bde 2023-02-11 03:54:48,655 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=51 updating hbase:meta row=7e59b0be260b97681ec2d05527ed1bde, regionState=CLOSED 2023-02-11 03:54:48,655 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676087672040.7e59b0be260b97681ec2d05527ed1bde.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1676087688655"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676087688655"}]},"ts":"1676087688655"} 2023-02-11 03:54:48,658 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=52, resume processing ppid=51 2023-02-11 03:54:48,658 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=52, ppid=51, state=SUCCESS; CloseRegionProcedure 7e59b0be260b97681ec2d05527ed1bde, server=jenkins-hbase4.apache.org,37011,1676087618428 in 162 msec 2023-02-11 03:54:48,658 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=51, ppid=50, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=7e59b0be260b97681ec2d05527ed1bde, REOPEN/MOVE; state=CLOSED, location=jenkins-hbase4.apache.org,37011,1676087618428; forceNewPlan=false, retain=true 2023-02-11 03:54:48,809 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=51 updating hbase:meta row=7e59b0be260b97681ec2d05527ed1bde, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:48,809 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676087672040.7e59b0be260b97681ec2d05527ed1bde.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1676087688809"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676087688809"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676087688809"}]},"ts":"1676087688809"} 2023-02-11 03:54:48,811 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=53, ppid=51, state=RUNNABLE; OpenRegionProcedure 7e59b0be260b97681ec2d05527ed1bde, server=jenkins-hbase4.apache.org,37011,1676087618428}] 2023-02-11 03:54:48,965 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676087672040.7e59b0be260b97681ec2d05527ed1bde. 2023-02-11 03:54:48,965 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 7e59b0be260b97681ec2d05527ed1bde, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676087672040.7e59b0be260b97681ec2d05527ed1bde.', STARTKEY => '', ENDKEY => ''} 2023-02-11 03:54:48,965 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 7e59b0be260b97681ec2d05527ed1bde 2023-02-11 03:54:48,965 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676087672040.7e59b0be260b97681ec2d05527ed1bde.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-11 03:54:48,965 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 7e59b0be260b97681ec2d05527ed1bde 2023-02-11 03:54:48,965 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 7e59b0be260b97681ec2d05527ed1bde 2023-02-11 03:54:48,966 INFO [StoreOpener-7e59b0be260b97681ec2d05527ed1bde-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 7e59b0be260b97681ec2d05527ed1bde 2023-02-11 03:54:48,967 DEBUG [StoreOpener-7e59b0be260b97681ec2d05527ed1bde-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/7e59b0be260b97681ec2d05527ed1bde/family1 2023-02-11 03:54:48,967 DEBUG [StoreOpener-7e59b0be260b97681ec2d05527ed1bde-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/7e59b0be260b97681ec2d05527ed1bde/family1 2023-02-11 03:54:48,968 INFO [StoreOpener-7e59b0be260b97681ec2d05527ed1bde-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 7e59b0be260b97681ec2d05527ed1bde columnFamilyName family1 2023-02-11 03:54:48,972 DEBUG [StoreOpener-7e59b0be260b97681ec2d05527ed1bde-1] regionserver.HStore(539): loaded hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/7e59b0be260b97681ec2d05527ed1bde/family1/77f4e1f915eb4dfebae97d0ac7c88d04_SeqId_53_ 2023-02-11 03:54:48,975 DEBUG [StoreOpener-7e59b0be260b97681ec2d05527ed1bde-1] regionserver.HStore(539): loaded hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/7e59b0be260b97681ec2d05527ed1bde/family1/fc95456c53b24e878864598023112023 2023-02-11 03:54:48,975 INFO [StoreOpener-7e59b0be260b97681ec2d05527ed1bde-1] regionserver.HStore(310): Store=7e59b0be260b97681ec2d05527ed1bde/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 03:54:48,976 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/7e59b0be260b97681ec2d05527ed1bde 2023-02-11 03:54:48,977 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/7e59b0be260b97681ec2d05527ed1bde 2023-02-11 03:54:48,980 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 7e59b0be260b97681ec2d05527ed1bde 2023-02-11 03:54:48,981 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 7e59b0be260b97681ec2d05527ed1bde; next sequenceid=56; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=74215945, jitterRate=0.105903759598732}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-11 03:54:48,981 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 7e59b0be260b97681ec2d05527ed1bde: Running coprocessor pre-open hook at 1676087688965Writing region info on filesystem at 1676087688965Initializing all the Stores at 1676087688966 (+1 ms)Instantiating store for column family {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} at 1676087688966Cleaning up temporary data from old regions at 1676087688978 (+12 ms)Cleaning up detritus from prior splits at 1676087688978Running coprocessor post-open hooks at 1676087688981 (+3 ms)Region opened successfully at 1676087688981 2023-02-11 03:54:48,981 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676087672040.7e59b0be260b97681ec2d05527ed1bde., pid=53, masterSystemTime=1676087688962 2023-02-11 03:54:48,983 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676087672040.7e59b0be260b97681ec2d05527ed1bde. 2023-02-11 03:54:48,983 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676087672040.7e59b0be260b97681ec2d05527ed1bde. 2023-02-11 03:54:48,983 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=51 updating hbase:meta row=7e59b0be260b97681ec2d05527ed1bde, regionState=OPEN, openSeqNum=56, regionLocation=jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:48,984 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676087672040.7e59b0be260b97681ec2d05527ed1bde.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1676087688983"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676087688983"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676087688983"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676087688983"}]},"ts":"1676087688983"} 2023-02-11 03:54:48,986 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=53, resume processing ppid=51 2023-02-11 03:54:48,986 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=53, ppid=51, state=SUCCESS; OpenRegionProcedure 7e59b0be260b97681ec2d05527ed1bde, server=jenkins-hbase4.apache.org,37011,1676087618428 in 175 msec 2023-02-11 03:54:48,987 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=51, resume processing ppid=50 2023-02-11 03:54:48,987 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=51, ppid=50, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=7e59b0be260b97681ec2d05527ed1bde, REOPEN/MOVE in 493 msec 2023-02-11 03:54:48,989 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=50, resume processing ppid=49 2023-02-11 03:54:48,989 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=50, ppid=49, state=SUCCESS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly in 498 msec 2023-02-11 03:54:48,991 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=49, state=SUCCESS; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly in 913 msec 2023-02-11 03:54:48,992 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] master.MasterRpcServices(1227): Checking to see if procedure is done pid=49 2023-02-11 03:54:48,993 INFO [Listener at localhost/46205] compactions.TestMobCompactor(278): alter status finished 2023-02-11 03:54:48,994 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=39921] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}] of the table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-11 03:54:48,995 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676087621935] procedure2.ProcedureExecutor(1029): Stored pid=54, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-02-11 03:54:48,995 DEBUG [PEWorker-2] locking.LockProcedure(309): LOCKED pid=54, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-02-11 03:54:48,996 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676087621935] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-11 03:54:48,996 INFO [PEWorker-2] procedure2.TimeoutExecutorThread(81): ADDED pid=54, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1676088288996 2023-02-11 03:54:48,996 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676087621935] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-02-11 03:54:48,997 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676087621935] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 4 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column 'family1' 2023-02-11 03:54:48,997 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676087621935] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-02-11 03:54:48,997 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676087621935] zookeeper.ReadOnlyZKClient(139): Connect 0x644ac847 to 127.0.0.1:53383 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-11 03:54:49,001 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676087621935] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@14fa9b97, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-11 03:54:49,001 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-11 03:54:49,001 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-11 03:54:49,001 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-11 03:54:49,001 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-11 03:54:49,001 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-11 03:54:49,001 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-11 03:54:49,001 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-11 03:54:49,002 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-11 03:54:49,002 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676087621935] compactions.PartitionedMobCompactor(354): After compaction, there are 4 mob files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-02-11 03:54:49,002 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676087621935] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-11 03:54:49,003 DEBUG [PEWorker-1] locking.LockProcedure(242): UNLOCKED pid=54, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-02-11 03:54:49,004 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=54, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE in 9 msec 2023-02-11 03:54:49,068 INFO [Listener at localhost/46205] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly Thread=241 (was 267), OpenFileDescriptor=608 (was 645), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=159 (was 195), ProcessCount=168 (was 168), AvailableMemoryMB=3126 (was 3174) 2023-02-11 03:54:49,081 INFO [Listener at localhost/46205] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily Thread=241, OpenFileDescriptor=608, MaxFileDescriptor=60000, SystemLoadAverage=159, ProcessCount=168, AvailableMemoryMB=3126 2023-02-11 03:54:49,083 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] master.HMaster$4(2102): Client=jenkins//172.31.14.131 create 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} 2023-02-11 03:54:49,084 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] procedure2.ProcedureExecutor(1029): Stored pid=55, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:49,085 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=55, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily execute state=CREATE_TABLE_PRE_OPERATION 2023-02-11 03:54:49,085 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] master.MasterRpcServices(697): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily" procId is: 55 2023-02-11 03:54:49,086 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=55, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-11 03:54:49,086 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] master.MasterRpcServices(1227): Checking to see if procedure is done pid=55 2023-02-11 03:54:49,087 DEBUG [HFileArchiver-10] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/614a4b4b2b88efbc57d9d64c1044323f 2023-02-11 03:54:49,087 DEBUG [HFileArchiver-10] backup.HFileArchiver(153): Directory hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/614a4b4b2b88efbc57d9d64c1044323f empty. 2023-02-11 03:54:49,088 DEBUG [HFileArchiver-10] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/614a4b4b2b88efbc57d9d64c1044323f 2023-02-11 03:54:49,088 DEBUG [PEWorker-3] procedure.DeleteTableProcedure(328): Archived testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily regions 2023-02-11 03:54:49,187 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] master.MasterRpcServices(1227): Checking to see if procedure is done pid=55 2023-02-11 03:54:49,388 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] master.MasterRpcServices(1227): Checking to see if procedure is done pid=55 2023-02-11 03:54:49,495 DEBUG [PEWorker-3] util.FSTableDescriptors(570): Wrote into hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/.tabledesc/.tableinfo.0000000001 2023-02-11 03:54:49,497 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(7671): creating {ENCODED => 614a4b4b2b88efbc57d9d64c1044323f, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676087689083.614a4b4b2b88efbc57d9d64c1044323f.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}, regionDir=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/.tmp 2023-02-11 03:54:49,689 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] master.MasterRpcServices(1227): Checking to see if procedure is done pid=55 2023-02-11 03:54:49,902 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(865): Instantiated testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676087689083.614a4b4b2b88efbc57d9d64c1044323f.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-11 03:54:49,902 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(1603): Closing 614a4b4b2b88efbc57d9d64c1044323f, disabling compactions & flushes 2023-02-11 03:54:49,902 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(1625): Closing region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676087689083.614a4b4b2b88efbc57d9d64c1044323f. 2023-02-11 03:54:49,902 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676087689083.614a4b4b2b88efbc57d9d64c1044323f. 2023-02-11 03:54:49,902 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(1713): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676087689083.614a4b4b2b88efbc57d9d64c1044323f. after waiting 0 ms 2023-02-11 03:54:49,902 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(1723): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676087689083.614a4b4b2b88efbc57d9d64c1044323f. 2023-02-11 03:54:49,902 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(1837): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676087689083.614a4b4b2b88efbc57d9d64c1044323f. 2023-02-11 03:54:49,902 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(1557): Region close journal for 614a4b4b2b88efbc57d9d64c1044323f: Waiting for close lock at 1676087689902Disabling compacts and flushes for region at 1676087689902Disabling writes for close at 1676087689902Writing region close event to WAL at 1676087689902Closed at 1676087689902 2023-02-11 03:54:49,905 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=55, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily execute state=CREATE_TABLE_ADD_TO_META 2023-02-11 03:54:49,905 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676087689083.614a4b4b2b88efbc57d9d64c1044323f.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1676087689905"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676087689905"}]},"ts":"1676087689905"} 2023-02-11 03:54:49,907 INFO [PEWorker-3] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-11 03:54:49,907 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=55, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-11 03:54:49,907 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676087689907"}]},"ts":"1676087689907"} 2023-02-11 03:54:49,908 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, state=ENABLING in hbase:meta 2023-02-11 03:54:49,911 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=56, ppid=55, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=614a4b4b2b88efbc57d9d64c1044323f, ASSIGN}] 2023-02-11 03:54:49,912 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=56, ppid=55, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=614a4b4b2b88efbc57d9d64c1044323f, ASSIGN 2023-02-11 03:54:49,913 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=56, ppid=55, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=614a4b4b2b88efbc57d9d64c1044323f, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,37011,1676087618428; forceNewPlan=false, retain=false 2023-02-11 03:54:50,063 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=56 updating hbase:meta row=614a4b4b2b88efbc57d9d64c1044323f, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:50,064 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676087689083.614a4b4b2b88efbc57d9d64c1044323f.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1676087690063"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676087690063"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676087690063"}]},"ts":"1676087690063"} 2023-02-11 03:54:50,065 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=57, ppid=56, state=RUNNABLE; OpenRegionProcedure 614a4b4b2b88efbc57d9d64c1044323f, server=jenkins-hbase4.apache.org,37011,1676087618428}] 2023-02-11 03:54:50,189 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] master.MasterRpcServices(1227): Checking to see if procedure is done pid=55 2023-02-11 03:54:50,219 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676087689083.614a4b4b2b88efbc57d9d64c1044323f. 2023-02-11 03:54:50,219 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 614a4b4b2b88efbc57d9d64c1044323f, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676087689083.614a4b4b2b88efbc57d9d64c1044323f.', STARTKEY => '', ENDKEY => ''} 2023-02-11 03:54:50,220 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 614a4b4b2b88efbc57d9d64c1044323f 2023-02-11 03:54:50,220 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676087689083.614a4b4b2b88efbc57d9d64c1044323f.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-11 03:54:50,220 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 614a4b4b2b88efbc57d9d64c1044323f 2023-02-11 03:54:50,220 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 614a4b4b2b88efbc57d9d64c1044323f 2023-02-11 03:54:50,221 INFO [StoreOpener-614a4b4b2b88efbc57d9d64c1044323f-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 614a4b4b2b88efbc57d9d64c1044323f 2023-02-11 03:54:50,222 DEBUG [StoreOpener-614a4b4b2b88efbc57d9d64c1044323f-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/614a4b4b2b88efbc57d9d64c1044323f/family1 2023-02-11 03:54:50,222 DEBUG [StoreOpener-614a4b4b2b88efbc57d9d64c1044323f-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/614a4b4b2b88efbc57d9d64c1044323f/family1 2023-02-11 03:54:50,222 INFO [StoreOpener-614a4b4b2b88efbc57d9d64c1044323f-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 614a4b4b2b88efbc57d9d64c1044323f columnFamilyName family1 2023-02-11 03:54:50,224 INFO [StoreOpener-614a4b4b2b88efbc57d9d64c1044323f-1] regionserver.HStore(310): Store=614a4b4b2b88efbc57d9d64c1044323f/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 03:54:50,224 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/614a4b4b2b88efbc57d9d64c1044323f 2023-02-11 03:54:50,225 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/614a4b4b2b88efbc57d9d64c1044323f 2023-02-11 03:54:50,227 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 614a4b4b2b88efbc57d9d64c1044323f 2023-02-11 03:54:50,229 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/614a4b4b2b88efbc57d9d64c1044323f/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-11 03:54:50,229 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 614a4b4b2b88efbc57d9d64c1044323f; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=69777180, jitterRate=0.0397610068321228}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-11 03:54:50,230 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 614a4b4b2b88efbc57d9d64c1044323f: Running coprocessor pre-open hook at 1676087690220Writing region info on filesystem at 1676087690220Initializing all the Stores at 1676087690220Instantiating store for column family {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} at 1676087690220Cleaning up temporary data from old regions at 1676087690225 (+5 ms)Cleaning up detritus from prior splits at 1676087690225Running coprocessor post-open hooks at 1676087690230 (+5 ms)Region opened successfully at 1676087690230 2023-02-11 03:54:50,230 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676087689083.614a4b4b2b88efbc57d9d64c1044323f., pid=57, masterSystemTime=1676087690217 2023-02-11 03:54:50,231 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676087689083.614a4b4b2b88efbc57d9d64c1044323f. 2023-02-11 03:54:50,232 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676087689083.614a4b4b2b88efbc57d9d64c1044323f. 2023-02-11 03:54:50,232 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=56 updating hbase:meta row=614a4b4b2b88efbc57d9d64c1044323f, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:50,232 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676087689083.614a4b4b2b88efbc57d9d64c1044323f.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1676087690232"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676087690232"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676087690232"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676087690232"}]},"ts":"1676087690232"} 2023-02-11 03:54:50,235 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=57, resume processing ppid=56 2023-02-11 03:54:50,235 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=57, ppid=56, state=SUCCESS; OpenRegionProcedure 614a4b4b2b88efbc57d9d64c1044323f, server=jenkins-hbase4.apache.org,37011,1676087618428 in 168 msec 2023-02-11 03:54:50,236 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=56, resume processing ppid=55 2023-02-11 03:54:50,236 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=56, ppid=55, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=614a4b4b2b88efbc57d9d64c1044323f, ASSIGN in 325 msec 2023-02-11 03:54:50,237 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=55, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-11 03:54:50,237 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676087690237"}]},"ts":"1676087690237"} 2023-02-11 03:54:50,238 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, state=ENABLED in hbase:meta 2023-02-11 03:54:50,239 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=55, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily execute state=CREATE_TABLE_POST_OPERATION 2023-02-11 03:54:50,240 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=55, state=SUCCESS; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily in 1.1560 sec 2023-02-11 03:54:51,190 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] master.MasterRpcServices(1227): Checking to see if procedure is done pid=55 2023-02-11 03:54:51,191 INFO [Listener at localhost/46205] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, procId: 55 completed 2023-02-11 03:54:51,209 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 03:54:51,211 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:51,211 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-02-11 03:54:51,211 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:54:51,211 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-02-11 03:54:51,211 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-02-11 03:54:51,211 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:51,212 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:51,214 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:51,214 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:51,214 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:51,214 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:51,214 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:51,215 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 03:54:51,215 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:51,215 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:51,215 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 03:54:51,215 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:51,215 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:51,216 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:51,216 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-02-11 03:54:51,216 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:54:51,216 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-02-11 03:54:51,216 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 03:54:51,216 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 03:54:51,216 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676087689083.614a4b4b2b88efbc57d9d64c1044323f. 2023-02-11 03:54:51,216 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676087689083.614a4b4b2b88efbc57d9d64c1044323f. started... 2023-02-11 03:54:51,217 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 614a4b4b2b88efbc57d9d64c1044323f 1/1 column families, dataSize=74.11 KB heapSize=148.69 KB 2023-02-11 03:54:51,641 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151130984a7d1aeb464af0b2519dba3ccaf447 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151130984a7d1aeb464af0b2519dba3ccaf447 2023-02-11 03:54:52,045 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=74.1 K, hasBloomFilter=true, into tmp file hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/614a4b4b2b88efbc57d9d64c1044323f/.tmp/family1/9824724851a8448685c3cdd9e40b73ab 2023-02-11 03:54:52,049 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/614a4b4b2b88efbc57d9d64c1044323f/.tmp/family1/9824724851a8448685c3cdd9e40b73ab as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/614a4b4b2b88efbc57d9d64c1044323f/family1/9824724851a8448685c3cdd9e40b73ab 2023-02-11 03:54:52,053 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/614a4b4b2b88efbc57d9d64c1044323f/family1/9824724851a8448685c3cdd9e40b73ab, entries=1000, sequenceid=5, filesize=190.5 K 2023-02-11 03:54:52,054 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~74.11 KB/75890, heapSize ~148.67 KB/152240, currentSize=0 B/0 for 614a4b4b2b88efbc57d9d64c1044323f in 837ms, sequenceid=5, compaction requested=false 2023-02-11 03:54:52,054 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-02-11 03:54:52,054 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 614a4b4b2b88efbc57d9d64c1044323f: 2023-02-11 03:54:52,054 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676087689083.614a4b4b2b88efbc57d9d64c1044323f. 2023-02-11 03:54:52,054 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 03:54:52,055 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 03:54:52,055 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:52,055 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-02-11 03:54:52,055 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37011,1676087618428' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-02-11 03:54:52,056 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:52,056 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:52,056 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:52,056 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:54:52,056 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:54:52,056 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:52,056 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 03:54:52,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:54:52,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:54:52,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:52,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:52,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:54:52,058 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37011,1676087618428' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-02-11 03:54:52,058 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@652b9bc7[Count = 0] remaining members to acquire global barrier 2023-02-11 03:54:52,058 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-02-11 03:54:52,058 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:52,059 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:52,059 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:52,059 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:52,059 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-02-11 03:54:52,059 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:52,059 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 03:54:52,059 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-02-11 03:54:52,059 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase4.apache.org,37011,1676087618428' in zk 2023-02-11 03:54:52,060 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:52,060 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-02-11 03:54:52,060 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:54:52,060 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:52,060 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:54:52,060 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:54:52,060 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-02-11 03:54:52,061 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:54:52,061 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:54:52,061 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:52,061 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:52,061 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:54:52,062 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:52,062 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:52,062 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase4.apache.org,37011,1676087618428': 2023-02-11 03:54:52,062 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37011,1676087618428' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-02-11 03:54:52,062 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-02-11 03:54:52,062 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 03:54:52,062 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 03:54:52,062 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:52,062 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 03:54:52,065 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:52,065 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:52,065 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:52,065 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:52,065 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:52,065 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:54:52,065 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:54:52,065 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:52,065 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:52,065 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:54:52,065 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:52,065 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:52,065 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:52,066 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:52,066 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:54:52,066 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:52,066 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:52,066 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:52,066 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:54:52,067 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:52,067 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:52,072 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:52,072 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:52,072 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:52,072 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:52,072 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-02-11 03:54:52,072 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 03:54:52,072 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:52,072 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:52,072 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:54:52,072 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:52,072 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-02-11 03:54:52,072 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily because node does not exist (not an error) 2023-02-11 03:54:52,072 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:52,072 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:52,072 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 03:54:52,073 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:52,073 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:52,073 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:52,073 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:52,173 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 03:54:52,173 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 03:54:52,177 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 03:54:52,178 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-02-11 03:54:52,179 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:52,179 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-02-11 03:54:52,179 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:54:52,179 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-02-11 03:54:52,179 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-02-11 03:54:52,179 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:52,179 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:52,180 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:52,180 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:52,180 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:52,180 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:52,180 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:52,181 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 03:54:52,181 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:52,181 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:52,181 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 03:54:52,181 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:52,181 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:52,181 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-02-11 03:54:52,181 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:52,181 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-02-11 03:54:52,182 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:54:52,182 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-02-11 03:54:52,182 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 03:54:52,182 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 03:54:52,182 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676087689083.614a4b4b2b88efbc57d9d64c1044323f. 2023-02-11 03:54:52,182 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676087689083.614a4b4b2b88efbc57d9d64c1044323f. started... 2023-02-11 03:54:52,182 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 614a4b4b2b88efbc57d9d64c1044323f 1/1 column families, dataSize=74 B heapSize=408 B 2023-02-11 03:54:52,320 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-11 03:54:52,596 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151128b71e1b0d76d448978901f9dbb635fc87 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151128b71e1b0d76d448978901f9dbb635fc87 2023-02-11 03:54:53,001 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=74, hasBloomFilter=true, into tmp file hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/614a4b4b2b88efbc57d9d64c1044323f/.tmp/family1/969d83483a884b32acfaccab43ab4139 2023-02-11 03:54:53,005 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/614a4b4b2b88efbc57d9d64c1044323f/.tmp/family1/969d83483a884b32acfaccab43ab4139 as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/614a4b4b2b88efbc57d9d64c1044323f/family1/969d83483a884b32acfaccab43ab4139 2023-02-11 03:54:53,009 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/614a4b4b2b88efbc57d9d64c1044323f/family1/969d83483a884b32acfaccab43ab4139, entries=1, sequenceid=9, filesize=5.0 K 2023-02-11 03:54:53,010 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~74 B/74, heapSize ~392 B/392, currentSize=0 B/0 for 614a4b4b2b88efbc57d9d64c1044323f in 828ms, sequenceid=9, compaction requested=false 2023-02-11 03:54:53,010 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 614a4b4b2b88efbc57d9d64c1044323f: 2023-02-11 03:54:53,010 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676087689083.614a4b4b2b88efbc57d9d64c1044323f. 2023-02-11 03:54:53,010 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 03:54:53,010 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 03:54:53,010 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:53,010 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-02-11 03:54:53,010 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37011,1676087618428' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-02-11 03:54:53,012 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:53,012 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:53,012 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:53,013 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:54:53,013 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:54:53,013 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:53,013 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 03:54:53,013 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:54:53,013 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:54:53,013 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:53,014 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:53,014 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:54:53,014 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37011,1676087618428' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-02-11 03:54:53,014 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@73a08642[Count = 0] remaining members to acquire global barrier 2023-02-11 03:54:53,014 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-02-11 03:54:53,014 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:53,015 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:53,015 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:53,015 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:53,015 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-02-11 03:54:53,015 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-02-11 03:54:53,015 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase4.apache.org,37011,1676087618428' in zk 2023-02-11 03:54:53,015 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:53,015 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 03:54:53,016 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:53,016 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-02-11 03:54:53,016 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:54:53,016 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:53,016 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:54:53,016 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:54:53,016 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-02-11 03:54:53,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:54:53,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:54:53,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:53,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:53,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:54:53,018 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:53,018 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:53,018 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase4.apache.org,37011,1676087618428': 2023-02-11 03:54:53,018 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37011,1676087618428' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-02-11 03:54:53,018 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-02-11 03:54:53,018 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 03:54:53,018 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 03:54:53,018 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:53,018 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 03:54:53,020 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:53,020 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:53,020 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:53,020 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:54:53,020 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:54:53,020 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:53,020 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:53,020 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:53,020 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:54:53,020 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:53,020 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:53,020 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:53,020 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:53,021 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:53,021 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:54:53,021 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:53,021 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:53,021 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:53,021 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:54:53,022 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:53,022 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:53,024 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:53,024 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:53,024 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:53,024 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:53,024 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:54:53,024 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-02-11 03:54:53,024 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 03:54:53,024 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:53,024 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:53,024 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:53,024 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:53,024 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:53,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:53,024 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-02-11 03:54:53,024 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 03:54:53,024 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:53,025 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:53,025 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:53,125 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 03:54:53,125 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 03:54:53,129 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 03:54:53,130 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-02-11 03:54:53,130 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:53,130 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-02-11 03:54:53,130 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:54:53,130 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-02-11 03:54:53,130 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-02-11 03:54:53,131 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:53,131 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:53,132 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:53,132 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:53,132 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:53,132 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:53,132 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:53,132 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 03:54:53,132 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:53,132 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:53,133 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 03:54:53,133 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:53,133 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:53,133 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-02-11 03:54:53,133 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:53,133 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-02-11 03:54:53,133 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:54:53,133 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-02-11 03:54:53,133 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 03:54:53,133 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 03:54:53,133 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676087689083.614a4b4b2b88efbc57d9d64c1044323f. 2023-02-11 03:54:53,133 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676087689083.614a4b4b2b88efbc57d9d64c1044323f. started... 2023-02-11 03:54:53,133 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 614a4b4b2b88efbc57d9d64c1044323f 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-11 03:54:53,547 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151201e3cdfa2e6ec047b584c1eefd9398d097 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151201e3cdfa2e6ec047b584c1eefd9398d097 2023-02-11 03:54:53,952 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/614a4b4b2b88efbc57d9d64c1044323f/.tmp/family1/601d6562a72a48aaa7d328507e2e4a94 2023-02-11 03:54:53,956 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/614a4b4b2b88efbc57d9d64c1044323f/.tmp/family1/601d6562a72a48aaa7d328507e2e4a94 as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/614a4b4b2b88efbc57d9d64c1044323f/family1/601d6562a72a48aaa7d328507e2e4a94 2023-02-11 03:54:53,960 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/614a4b4b2b88efbc57d9d64c1044323f/family1/601d6562a72a48aaa7d328507e2e4a94, entries=1, sequenceid=13, filesize=5.0 K 2023-02-11 03:54:53,961 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 614a4b4b2b88efbc57d9d64c1044323f in 828ms, sequenceid=13, compaction requested=true 2023-02-11 03:54:53,961 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 614a4b4b2b88efbc57d9d64c1044323f: 2023-02-11 03:54:53,961 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676087689083.614a4b4b2b88efbc57d9d64c1044323f. 2023-02-11 03:54:53,961 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 03:54:53,961 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 03:54:53,961 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:53,961 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-02-11 03:54:53,961 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37011,1676087618428' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-02-11 03:54:53,962 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:53,962 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:53,963 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:53,963 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:54:53,963 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:54:53,963 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:53,963 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 03:54:53,963 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:54:53,963 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:54:53,963 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:53,964 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:53,964 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:54:53,964 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37011,1676087618428' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-02-11 03:54:53,964 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@74e90d[Count = 0] remaining members to acquire global barrier 2023-02-11 03:54:53,964 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-02-11 03:54:53,964 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:53,966 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:53,966 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:53,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:53,966 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-02-11 03:54:53,966 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-02-11 03:54:53,966 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase4.apache.org,37011,1676087618428' in zk 2023-02-11 03:54:53,966 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:53,966 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 03:54:53,968 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-02-11 03:54:53,968 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:53,968 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:54:53,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:53,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:54:53,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:54:53,968 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-02-11 03:54:53,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:54:53,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:54:53,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:53,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:53,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:54:53,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:53,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:53,970 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase4.apache.org,37011,1676087618428': 2023-02-11 03:54:53,970 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37011,1676087618428' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-02-11 03:54:53,970 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-02-11 03:54:53,970 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 03:54:53,970 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 03:54:53,970 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:53,970 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 03:54:53,971 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:53,971 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:53,971 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:53,971 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:53,971 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:54:53,972 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:54:53,971 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:53,972 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:53,972 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:53,972 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:54:53,972 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:53,972 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:53,972 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:53,972 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:53,972 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:54:53,973 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:53,973 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:53,973 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:53,973 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:54:53,973 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:53,974 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:53,975 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:53,975 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:53,975 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:53,975 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:53,975 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:53,975 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:53,975 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-02-11 03:54:53,975 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 03:54:53,975 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:53,975 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:54:53,975 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:53,975 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:53,976 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:53,975 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:53,976 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-02-11 03:54:53,976 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 03:54:53,976 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:53,976 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:54,076 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 03:54:54,077 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 03:54:54,080 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 03:54:54,082 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-02-11 03:54:54,082 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:54,082 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-02-11 03:54:54,082 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:54:54,082 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-02-11 03:54:54,082 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-02-11 03:54:54,082 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:54,082 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:54,083 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:54,083 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:54,084 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:54,084 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:54,084 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:54,084 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 03:54:54,084 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:54,084 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:54,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 03:54:54,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:54,085 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:54,085 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-02-11 03:54:54,085 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:54,085 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-02-11 03:54:54,085 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:54:54,085 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-02-11 03:54:54,085 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 03:54:54,085 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 03:54:54,085 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676087689083.614a4b4b2b88efbc57d9d64c1044323f. 2023-02-11 03:54:54,085 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676087689083.614a4b4b2b88efbc57d9d64c1044323f. started... 2023-02-11 03:54:54,085 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 614a4b4b2b88efbc57d9d64c1044323f 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-11 03:54:54,098 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015120536f6952537dc4b37bb6e12cd74a2cf6f to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e2015120536f6952537dc4b37bb6e12cd74a2cf6f 2023-02-11 03:54:54,503 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/614a4b4b2b88efbc57d9d64c1044323f/.tmp/family1/7d5fe952ece94c4cbcf6e567be0c1641 2023-02-11 03:54:54,508 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/614a4b4b2b88efbc57d9d64c1044323f/.tmp/family1/7d5fe952ece94c4cbcf6e567be0c1641 as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/614a4b4b2b88efbc57d9d64c1044323f/family1/7d5fe952ece94c4cbcf6e567be0c1641 2023-02-11 03:54:54,513 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/614a4b4b2b88efbc57d9d64c1044323f/family1/7d5fe952ece94c4cbcf6e567be0c1641, entries=1, sequenceid=17, filesize=5.0 K 2023-02-11 03:54:54,513 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 614a4b4b2b88efbc57d9d64c1044323f in 428ms, sequenceid=17, compaction requested=true 2023-02-11 03:54:54,513 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 614a4b4b2b88efbc57d9d64c1044323f: 2023-02-11 03:54:54,513 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676087689083.614a4b4b2b88efbc57d9d64c1044323f. 2023-02-11 03:54:54,513 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 03:54:54,514 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 03:54:54,514 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:54,514 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-02-11 03:54:54,514 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37011,1676087618428' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-02-11 03:54:54,515 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:54,515 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:54,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:54,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:54:54,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:54:54,515 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:54,515 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 03:54:54,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:54:54,516 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:54:54,516 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:54,516 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:54,516 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:54:54,517 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37011,1676087618428' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-02-11 03:54:54,517 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@61189068[Count = 0] remaining members to acquire global barrier 2023-02-11 03:54:54,517 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-02-11 03:54:54,517 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:54,522 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:54,522 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:54,522 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:54,522 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-02-11 03:54:54,522 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-02-11 03:54:54,522 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase4.apache.org,37011,1676087618428' in zk 2023-02-11 03:54:54,522 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:54,522 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 03:54:54,527 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-02-11 03:54:54,527 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:54,527 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:54:54,527 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:54,527 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-02-11 03:54:54,527 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:54:54,527 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:54:54,528 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:54:54,528 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:54:54,528 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:54,528 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:54,528 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:54:54,529 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:54,529 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:54,529 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase4.apache.org,37011,1676087618428': 2023-02-11 03:54:54,529 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37011,1676087618428' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-02-11 03:54:54,529 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-02-11 03:54:54,529 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 03:54:54,529 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 03:54:54,529 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:54,529 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 03:54:54,531 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:54,531 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:54,531 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:54,531 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:54:54,531 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:54:54,531 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:54,531 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:54,531 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:54,531 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:54:54,531 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:54,531 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:54,531 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:54,532 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:54,532 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:54,532 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:54:54,532 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:54,532 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:54,532 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:54,533 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:54:54,533 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:54,533 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:54,534 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:54,534 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:54,534 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:54,534 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:54,534 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:54,534 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:54:54,534 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:54,534 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-02-11 03:54:54,534 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:54,535 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 03:54:54,535 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:54,534 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:54,534 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:54,535 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:54,535 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-02-11 03:54:54,535 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 03:54:54,535 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:54,535 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:54,635 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 03:54:54,636 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 03:54:54,639 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 03:54:54,640 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-02-11 03:54:54,641 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:54,641 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-02-11 03:54:54,641 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:54:54,641 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-02-11 03:54:54,641 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-02-11 03:54:54,641 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:54,641 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:54,642 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:54,642 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:54,642 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:54,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:54,642 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:54,642 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 03:54:54,643 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:54,643 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:54,643 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 03:54:54,643 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:54,643 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:54,643 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-02-11 03:54:54,643 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:54,643 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-02-11 03:54:54,644 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:54:54,644 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-02-11 03:54:54,644 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 03:54:54,644 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 03:54:54,644 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676087689083.614a4b4b2b88efbc57d9d64c1044323f. 2023-02-11 03:54:54,644 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676087689083.614a4b4b2b88efbc57d9d64c1044323f. started... 2023-02-11 03:54:54,644 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 614a4b4b2b88efbc57d9d64c1044323f 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-11 03:54:55,055 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151228d7eee638ea8b45578845c99c799736a1 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151228d7eee638ea8b45578845c99c799736a1 2023-02-11 03:54:55,460 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=21, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/614a4b4b2b88efbc57d9d64c1044323f/.tmp/family1/ae5281d8d9db44ff810fe3418c870289 2023-02-11 03:54:55,465 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/614a4b4b2b88efbc57d9d64c1044323f/.tmp/family1/ae5281d8d9db44ff810fe3418c870289 as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/614a4b4b2b88efbc57d9d64c1044323f/family1/ae5281d8d9db44ff810fe3418c870289 2023-02-11 03:54:55,468 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/614a4b4b2b88efbc57d9d64c1044323f/family1/ae5281d8d9db44ff810fe3418c870289, entries=1, sequenceid=21, filesize=5.0 K 2023-02-11 03:54:55,469 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 614a4b4b2b88efbc57d9d64c1044323f in 825ms, sequenceid=21, compaction requested=true 2023-02-11 03:54:55,469 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 614a4b4b2b88efbc57d9d64c1044323f: 2023-02-11 03:54:55,469 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676087689083.614a4b4b2b88efbc57d9d64c1044323f. 2023-02-11 03:54:55,469 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 03:54:55,469 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 03:54:55,469 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:55,470 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-02-11 03:54:55,470 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37011,1676087618428' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-02-11 03:54:55,472 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:55,472 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:55,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:55,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:54:55,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:54:55,473 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:55,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:54:55,473 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 03:54:55,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:54:55,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:55,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:55,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:54:55,474 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37011,1676087618428' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-02-11 03:54:55,474 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@58d4ca2f[Count = 0] remaining members to acquire global barrier 2023-02-11 03:54:55,474 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-02-11 03:54:55,474 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:55,475 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:55,475 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:55,475 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:55,475 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-02-11 03:54:55,475 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-02-11 03:54:55,475 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase4.apache.org,37011,1676087618428' in zk 2023-02-11 03:54:55,475 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:55,475 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 03:54:55,476 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-02-11 03:54:55,476 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:55,476 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:54:55,476 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:55,476 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:54:55,476 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-02-11 03:54:55,476 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:54:55,476 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:54:55,477 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:54:55,477 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:55,477 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:55,477 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:54:55,478 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:55,478 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:55,478 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase4.apache.org,37011,1676087618428': 2023-02-11 03:54:55,478 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37011,1676087618428' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-02-11 03:54:55,478 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-02-11 03:54:55,478 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 03:54:55,478 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 03:54:55,478 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:55,478 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 03:54:55,480 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:55,480 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:55,480 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:55,480 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:55,480 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:54:55,480 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:54:55,480 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:55,480 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:55,480 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:54:55,480 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:55,480 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:55,480 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:55,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:55,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:55,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:54:55,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:55,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:55,481 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:55,482 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:54:55,482 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:55,482 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:55,484 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:55,484 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:55,484 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:55,484 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:55,484 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:54:55,484 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:55,484 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:55,484 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily because node does not exist (not an error) 2023-02-11 03:54:55,484 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-02-11 03:54:55,485 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 03:54:55,484 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:55,485 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:55,485 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:55,485 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:55,485 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:55,485 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-02-11 03:54:55,485 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:55,485 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:55,485 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 03:54:55,585 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 03:54:55,586 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 03:54:55,590 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 03:54:55,591 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-02-11 03:54:55,591 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:55,591 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-02-11 03:54:55,591 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:54:55,591 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-02-11 03:54:55,591 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-02-11 03:54:55,592 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:55,592 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:55,593 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:55,593 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:55,593 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:55,593 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:55,593 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:55,593 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 03:54:55,593 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:55,594 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:55,594 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 03:54:55,594 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:55,594 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:55,595 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-02-11 03:54:55,595 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:55,595 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-02-11 03:54:55,595 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:54:55,595 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-02-11 03:54:55,595 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 03:54:55,595 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 03:54:55,595 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676087689083.614a4b4b2b88efbc57d9d64c1044323f. 2023-02-11 03:54:55,595 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676087689083.614a4b4b2b88efbc57d9d64c1044323f. started... 2023-02-11 03:54:55,595 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 614a4b4b2b88efbc57d9d64c1044323f 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-11 03:54:56,006 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512315446b1245496496ab72413015d087f07 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201512315446b1245496496ab72413015d087f07 2023-02-11 03:54:56,411 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=25, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/614a4b4b2b88efbc57d9d64c1044323f/.tmp/family1/2b3a885b0f5649b4a13ce1e8c2482c4f 2023-02-11 03:54:56,416 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/614a4b4b2b88efbc57d9d64c1044323f/.tmp/family1/2b3a885b0f5649b4a13ce1e8c2482c4f as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/614a4b4b2b88efbc57d9d64c1044323f/family1/2b3a885b0f5649b4a13ce1e8c2482c4f 2023-02-11 03:54:56,420 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/614a4b4b2b88efbc57d9d64c1044323f/family1/2b3a885b0f5649b4a13ce1e8c2482c4f, entries=1, sequenceid=25, filesize=5.0 K 2023-02-11 03:54:56,421 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 614a4b4b2b88efbc57d9d64c1044323f in 826ms, sequenceid=25, compaction requested=true 2023-02-11 03:54:56,421 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 614a4b4b2b88efbc57d9d64c1044323f: 2023-02-11 03:54:56,421 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676087689083.614a4b4b2b88efbc57d9d64c1044323f. 2023-02-11 03:54:56,421 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 03:54:56,421 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 03:54:56,421 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:56,421 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-02-11 03:54:56,421 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37011,1676087618428' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-02-11 03:54:56,422 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:56,422 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:56,422 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:56,422 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:54:56,422 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:54:56,423 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:56,423 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 03:54:56,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:54:56,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:54:56,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:56,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:56,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:54:56,424 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37011,1676087618428' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-02-11 03:54:56,424 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2fb1c5e1[Count = 0] remaining members to acquire global barrier 2023-02-11 03:54:56,424 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-02-11 03:54:56,424 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:56,425 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:56,425 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:56,425 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:56,425 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-02-11 03:54:56,425 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-02-11 03:54:56,425 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase4.apache.org,37011,1676087618428' in zk 2023-02-11 03:54:56,425 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:56,425 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 03:54:56,427 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-02-11 03:54:56,427 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:56,427 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:54:56,427 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:56,427 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:54:56,427 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:54:56,427 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-02-11 03:54:56,428 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:54:56,428 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:54:56,428 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:56,428 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:56,428 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:54:56,429 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:56,429 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:56,429 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase4.apache.org,37011,1676087618428': 2023-02-11 03:54:56,429 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37011,1676087618428' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-02-11 03:54:56,429 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-02-11 03:54:56,429 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 03:54:56,429 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 03:54:56,429 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:56,429 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 03:54:56,430 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:56,431 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:56,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:56,431 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:56,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:54:56,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:54:56,431 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:56,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:56,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:54:56,431 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:56,431 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:56,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:56,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:56,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:56,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:54:56,432 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:56,432 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:56,432 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:56,432 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:54:56,432 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:56,433 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:56,436 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:56,436 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:56,436 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:56,436 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:56,436 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:56,436 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-02-11 03:54:56,436 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily because node does not exist (not an error) 2023-02-11 03:54:56,436 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:56,437 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 03:54:56,436 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:56,436 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:54:56,437 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:56,437 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:56,437 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:56,437 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:56,437 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-02-11 03:54:56,437 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 03:54:56,437 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:56,437 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:56,537 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 03:54:56,538 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 03:54:56,541 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 03:54:56,543 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-02-11 03:54:56,543 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:56,543 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-02-11 03:54:56,543 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:54:56,543 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-02-11 03:54:56,543 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-02-11 03:54:56,543 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:56,543 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:56,548 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:56,548 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:56,548 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:56,548 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:56,548 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:56,548 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 03:54:56,548 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:56,549 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:56,549 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 03:54:56,549 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:56,549 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:56,549 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-02-11 03:54:56,549 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:56,549 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-02-11 03:54:56,549 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:54:56,549 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-02-11 03:54:56,549 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 03:54:56,549 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 03:54:56,549 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676087689083.614a4b4b2b88efbc57d9d64c1044323f. 2023-02-11 03:54:56,549 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676087689083.614a4b4b2b88efbc57d9d64c1044323f. started... 2023-02-11 03:54:56,550 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 614a4b4b2b88efbc57d9d64c1044323f 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-11 03:54:56,962 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201601014c4a5b1ab9c74800b0c69eb1234e8bb9 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201601014c4a5b1ab9c74800b0c69eb1234e8bb9 2023-02-11 03:54:57,368 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=29, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/614a4b4b2b88efbc57d9d64c1044323f/.tmp/family1/af123cf8b5d04e91b17b9e21cbbddcfd 2023-02-11 03:54:57,373 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/614a4b4b2b88efbc57d9d64c1044323f/.tmp/family1/af123cf8b5d04e91b17b9e21cbbddcfd as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/614a4b4b2b88efbc57d9d64c1044323f/family1/af123cf8b5d04e91b17b9e21cbbddcfd 2023-02-11 03:54:57,376 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/614a4b4b2b88efbc57d9d64c1044323f/family1/af123cf8b5d04e91b17b9e21cbbddcfd, entries=1, sequenceid=29, filesize=5.0 K 2023-02-11 03:54:57,377 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 614a4b4b2b88efbc57d9d64c1044323f in 827ms, sequenceid=29, compaction requested=true 2023-02-11 03:54:57,377 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 614a4b4b2b88efbc57d9d64c1044323f: 2023-02-11 03:54:57,377 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676087689083.614a4b4b2b88efbc57d9d64c1044323f. 2023-02-11 03:54:57,377 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 03:54:57,377 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 03:54:57,377 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:57,378 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-02-11 03:54:57,378 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37011,1676087618428' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-02-11 03:54:57,379 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:57,379 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:57,379 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:57,379 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:54:57,379 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:54:57,379 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:57,379 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 03:54:57,379 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:54:57,380 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:54:57,380 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:57,380 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:57,380 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:54:57,380 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37011,1676087618428' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-02-11 03:54:57,381 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@8aa6d9b[Count = 0] remaining members to acquire global barrier 2023-02-11 03:54:57,381 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-02-11 03:54:57,381 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:57,383 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:57,383 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:57,383 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:57,383 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-02-11 03:54:57,383 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-02-11 03:54:57,383 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:57,383 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 03:54:57,383 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase4.apache.org,37011,1676087618428' in zk 2023-02-11 03:54:57,384 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:57,384 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-02-11 03:54:57,385 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:54:57,385 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:57,385 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:54:57,385 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:54:57,385 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-02-11 03:54:57,385 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:54:57,385 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:54:57,385 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:57,386 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:57,386 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:54:57,386 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:57,386 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:57,386 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase4.apache.org,37011,1676087618428': 2023-02-11 03:54:57,386 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37011,1676087618428' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-02-11 03:54:57,386 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-02-11 03:54:57,387 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 03:54:57,387 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 03:54:57,387 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:57,387 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 03:54:57,388 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:57,388 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:57,388 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:57,388 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:54:57,388 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:54:57,388 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:57,388 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:57,388 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:57,389 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:54:57,389 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:57,389 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:57,389 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:57,389 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:57,389 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:57,389 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:54:57,389 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:57,390 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:57,390 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:57,390 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:54:57,390 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:57,391 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:57,392 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:57,392 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:57,392 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:57,392 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:57,392 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:57,392 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:54:57,392 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:57,392 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:57,392 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-02-11 03:54:57,392 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 03:54:57,392 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:57,392 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:57,392 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:57,392 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:57,392 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-02-11 03:54:57,392 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 03:54:57,392 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:57,393 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:57,493 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 03:54:57,493 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 03:54:57,497 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 03:54:57,499 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-02-11 03:54:57,499 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:57,499 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-02-11 03:54:57,499 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:54:57,500 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-02-11 03:54:57,500 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-02-11 03:54:57,500 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:57,500 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:57,502 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:57,502 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:57,502 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:57,502 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:57,503 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:57,503 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 03:54:57,503 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:57,503 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:57,503 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 03:54:57,503 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:57,503 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:57,504 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-02-11 03:54:57,504 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:57,504 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-02-11 03:54:57,504 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:54:57,504 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-02-11 03:54:57,504 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 03:54:57,504 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 03:54:57,504 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676087689083.614a4b4b2b88efbc57d9d64c1044323f. 2023-02-11 03:54:57,504 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676087689083.614a4b4b2b88efbc57d9d64c1044323f. started... 2023-02-11 03:54:57,504 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 614a4b4b2b88efbc57d9d64c1044323f 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-11 03:54:57,921 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201601037098c7fe2db6417eb0fbb457d674a234 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201601037098c7fe2db6417eb0fbb457d674a234 2023-02-11 03:54:58,326 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=33, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/614a4b4b2b88efbc57d9d64c1044323f/.tmp/family1/30936f2ca08942579f4e4582ffd95218 2023-02-11 03:54:58,331 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/614a4b4b2b88efbc57d9d64c1044323f/.tmp/family1/30936f2ca08942579f4e4582ffd95218 as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/614a4b4b2b88efbc57d9d64c1044323f/family1/30936f2ca08942579f4e4582ffd95218 2023-02-11 03:54:58,335 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/614a4b4b2b88efbc57d9d64c1044323f/family1/30936f2ca08942579f4e4582ffd95218, entries=1, sequenceid=33, filesize=5.0 K 2023-02-11 03:54:58,336 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 614a4b4b2b88efbc57d9d64c1044323f in 832ms, sequenceid=33, compaction requested=true 2023-02-11 03:54:58,336 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 614a4b4b2b88efbc57d9d64c1044323f: 2023-02-11 03:54:58,336 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676087689083.614a4b4b2b88efbc57d9d64c1044323f. 2023-02-11 03:54:58,336 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 03:54:58,337 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 03:54:58,337 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:58,337 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-02-11 03:54:58,337 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37011,1676087618428' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-02-11 03:54:58,338 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:58,338 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:58,338 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:58,338 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:54:58,338 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:54:58,339 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:58,339 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:54:58,339 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 03:54:58,339 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:54:58,339 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:58,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:58,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:54:58,340 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37011,1676087618428' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-02-11 03:54:58,340 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2d943736[Count = 0] remaining members to acquire global barrier 2023-02-11 03:54:58,340 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-02-11 03:54:58,340 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:58,341 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:58,342 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:58,342 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:58,342 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-02-11 03:54:58,342 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-02-11 03:54:58,342 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase4.apache.org,37011,1676087618428' in zk 2023-02-11 03:54:58,342 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:58,342 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 03:54:58,343 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-02-11 03:54:58,343 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:58,343 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:54:58,343 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:58,343 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:54:58,343 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:54:58,343 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-02-11 03:54:58,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:54:58,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:54:58,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:58,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:58,345 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:54:58,345 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:58,345 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:58,346 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase4.apache.org,37011,1676087618428': 2023-02-11 03:54:58,346 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37011,1676087618428' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-02-11 03:54:58,346 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-02-11 03:54:58,346 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 03:54:58,346 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 03:54:58,346 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:58,346 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 03:54:58,348 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:58,348 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:58,348 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:58,348 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:58,348 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:58,348 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:58,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:54:58,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:54:58,349 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:58,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:58,349 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:58,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:54:58,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:58,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:58,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:54:58,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:58,350 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:58,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:58,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:54:58,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:58,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:58,353 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:58,353 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:58,353 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:54:58,353 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:58,353 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-02-11 03:54:58,353 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 03:54:58,353 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:58,353 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:58,353 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily because node does not exist (not an error) 2023-02-11 03:54:58,353 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:58,353 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-02-11 03:54:58,353 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 03:54:58,353 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:58,353 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:58,354 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:58,354 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:58,354 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:58,354 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:58,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:58,454 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 03:54:58,455 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 03:54:58,458 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 03:54:58,460 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-02-11 03:54:58,460 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:58,460 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-02-11 03:54:58,460 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:54:58,460 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-02-11 03:54:58,460 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-02-11 03:54:58,461 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:58,461 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:58,462 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:58,462 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:58,462 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:58,462 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:58,462 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:58,462 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 03:54:58,463 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:58,463 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:58,463 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 03:54:58,463 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:58,463 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:58,463 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-02-11 03:54:58,463 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:58,463 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-02-11 03:54:58,464 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:54:58,464 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-02-11 03:54:58,464 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 03:54:58,464 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 03:54:58,464 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676087689083.614a4b4b2b88efbc57d9d64c1044323f. 2023-02-11 03:54:58,464 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676087689083.614a4b4b2b88efbc57d9d64c1044323f. started... 2023-02-11 03:54:58,464 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 614a4b4b2b88efbc57d9d64c1044323f 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-11 03:54:58,876 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201509079bcf0d0f47b24206a5b599594bd7fd15 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201509079bcf0d0f47b24206a5b599594bd7fd15 2023-02-11 03:54:59,281 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=37, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/614a4b4b2b88efbc57d9d64c1044323f/.tmp/family1/d2c3624580d544e2a50bfa271c54f9aa 2023-02-11 03:54:59,286 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/614a4b4b2b88efbc57d9d64c1044323f/.tmp/family1/d2c3624580d544e2a50bfa271c54f9aa as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/614a4b4b2b88efbc57d9d64c1044323f/family1/d2c3624580d544e2a50bfa271c54f9aa 2023-02-11 03:54:59,289 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/614a4b4b2b88efbc57d9d64c1044323f/family1/d2c3624580d544e2a50bfa271c54f9aa, entries=1, sequenceid=37, filesize=5.0 K 2023-02-11 03:54:59,290 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 614a4b4b2b88efbc57d9d64c1044323f in 826ms, sequenceid=37, compaction requested=true 2023-02-11 03:54:59,290 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 614a4b4b2b88efbc57d9d64c1044323f: 2023-02-11 03:54:59,290 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676087689083.614a4b4b2b88efbc57d9d64c1044323f. 2023-02-11 03:54:59,290 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 03:54:59,290 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 03:54:59,290 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:59,290 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-02-11 03:54:59,290 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37011,1676087618428' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-02-11 03:54:59,292 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:59,292 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:59,292 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:59,292 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:54:59,292 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:54:59,292 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:59,292 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 03:54:59,293 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:54:59,293 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:54:59,293 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:59,293 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:59,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:54:59,294 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37011,1676087618428' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-02-11 03:54:59,294 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@53ef0519[Count = 0] remaining members to acquire global barrier 2023-02-11 03:54:59,294 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-02-11 03:54:59,294 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:59,295 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:59,295 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:59,295 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:59,295 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-02-11 03:54:59,295 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-02-11 03:54:59,295 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:59,295 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 03:54:59,295 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase4.apache.org,37011,1676087618428' in zk 2023-02-11 03:54:59,296 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-02-11 03:54:59,296 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:54:59,296 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:59,296 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-02-11 03:54:59,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:59,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:54:59,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:54:59,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:54:59,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:54:59,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:59,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:59,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:54:59,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:59,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:59,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase4.apache.org,37011,1676087618428': 2023-02-11 03:54:59,298 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37011,1676087618428' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-02-11 03:54:59,298 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-02-11 03:54:59,298 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 03:54:59,299 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 03:54:59,299 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:59,299 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 03:54:59,300 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:59,300 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:59,300 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:59,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:59,300 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:59,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:59,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:54:59,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:54:59,300 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:59,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:59,300 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:59,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:54:59,301 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:59,301 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:59,301 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:54:59,301 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:59,301 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:59,302 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:59,302 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:54:59,302 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:59,302 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:59,305 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:59,305 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:59,305 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-02-11 03:54:59,305 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 03:54:59,305 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:59,305 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:54:59,305 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:54:59,305 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:59,305 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:59,305 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:59,306 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-02-11 03:54:59,306 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 03:54:59,306 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:54:59,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:54:59,306 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:59,306 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:59,306 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:59,306 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:59,406 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 03:54:59,407 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 03:54:59,410 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 03:54:59,411 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-02-11 03:54:59,412 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:59,412 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-02-11 03:54:59,412 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:54:59,412 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-02-11 03:54:59,412 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-02-11 03:54:59,412 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:59,412 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:59,413 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:54:59,413 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:59,413 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:54:59,413 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:54:59,414 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:54:59,414 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 03:54:59,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:59,414 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:59,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 03:54:59,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:59,414 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:59,415 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-02-11 03:54:59,415 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:54:59,415 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-02-11 03:54:59,415 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:54:59,415 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-02-11 03:54:59,415 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 03:54:59,415 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 03:54:59,415 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676087689083.614a4b4b2b88efbc57d9d64c1044323f. 2023-02-11 03:54:59,415 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676087689083.614a4b4b2b88efbc57d9d64c1044323f. started... 2023-02-11 03:54:59,415 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 614a4b4b2b88efbc57d9d64c1044323f 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-11 03:54:59,830 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201511209f41ce1c7a524ab7ad116d13285b5fe1 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201511209f41ce1c7a524ab7ad116d13285b5fe1 2023-02-11 03:55:00,235 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=41, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/614a4b4b2b88efbc57d9d64c1044323f/.tmp/family1/a0a87891a00b48ad87a274cb6061f4fc 2023-02-11 03:55:00,239 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/614a4b4b2b88efbc57d9d64c1044323f/.tmp/family1/a0a87891a00b48ad87a274cb6061f4fc as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/614a4b4b2b88efbc57d9d64c1044323f/family1/a0a87891a00b48ad87a274cb6061f4fc 2023-02-11 03:55:00,243 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/614a4b4b2b88efbc57d9d64c1044323f/family1/a0a87891a00b48ad87a274cb6061f4fc, entries=1, sequenceid=41, filesize=5.0 K 2023-02-11 03:55:00,243 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 614a4b4b2b88efbc57d9d64c1044323f in 828ms, sequenceid=41, compaction requested=true 2023-02-11 03:55:00,243 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 614a4b4b2b88efbc57d9d64c1044323f: 2023-02-11 03:55:00,244 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676087689083.614a4b4b2b88efbc57d9d64c1044323f. 2023-02-11 03:55:00,244 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 03:55:00,244 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 03:55:00,244 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:55:00,244 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-02-11 03:55:00,244 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37011,1676087618428' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-02-11 03:55:00,245 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:55:00,245 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:55:00,245 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:55:00,245 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:55:00,245 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:55:00,245 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:55:00,245 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 03:55:00,245 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:55:00,246 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:55:00,246 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:55:00,246 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:55:00,246 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:55:00,247 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37011,1676087618428' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-02-11 03:55:00,247 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@72504c48[Count = 0] remaining members to acquire global barrier 2023-02-11 03:55:00,247 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-02-11 03:55:00,247 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:55:00,248 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:55:00,248 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:55:00,248 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:55:00,248 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-02-11 03:55:00,248 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-02-11 03:55:00,248 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase4.apache.org,37011,1676087618428' in zk 2023-02-11 03:55:00,248 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:55:00,248 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 03:55:00,249 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-02-11 03:55:00,249 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:55:00,249 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:55:00,249 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-02-11 03:55:00,249 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:55:00,249 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:55:00,249 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:55:00,250 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:55:00,250 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:55:00,250 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:55:00,250 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:55:00,250 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:55:00,251 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:55:00,251 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:55:00,251 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase4.apache.org,37011,1676087618428': 2023-02-11 03:55:00,251 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37011,1676087618428' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-02-11 03:55:00,251 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-02-11 03:55:00,251 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 03:55:00,251 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 03:55:00,251 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:55:00,251 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 03:55:00,254 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:55:00,254 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:55:00,254 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:55:00,254 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:55:00,254 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:55:00,254 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:55:00,254 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:55:00,254 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:55:00,254 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:55:00,254 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:55:00,254 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:55:00,254 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:55:00,255 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:55:00,255 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:55:00,255 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:55:00,255 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:55:00,255 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:55:00,256 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:55:00,256 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:55:00,256 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:55:00,256 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:55:00,257 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:55:00,258 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:55:00,258 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:55:00,258 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:55:00,258 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-02-11 03:55:00,258 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 03:55:00,258 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:55:00,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:55:00,258 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:55:00,258 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:55:00,258 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:55:00,258 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-02-11 03:55:00,258 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:55:00,258 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 03:55:00,258 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:55:00,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:55:00,258 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:55:00,258 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:55:00,358 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 03:55:00,359 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 03:55:00,360 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=39921] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}] of the table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:55:00,361 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676087621935] procedure2.ProcedureExecutor(1029): Stored pid=58, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-02-11 03:55:00,362 DEBUG [PEWorker-5] locking.LockProcedure(309): LOCKED pid=58, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-02-11 03:55:00,363 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676087621935] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-11 03:55:00,363 INFO [PEWorker-5] procedure2.TimeoutExecutorThread(81): ADDED pid=58, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1676088300363 2023-02-11 03:55:00,365 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676087621935] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-02-11 03:55:00,366 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676087621935] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 10 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column 'family1' 2023-02-11 03:55:00,366 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676087621935] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-02-11 03:55:00,367 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676087621935] zookeeper.ReadOnlyZKClient(139): Connect 0x48890c4d to 127.0.0.1:53383 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-11 03:55:00,370 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676087621935] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@1b40f432, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-11 03:55:00,371 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-11 03:55:01,203 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160103c561190f6a584436a91b79b76c1495e3 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20160103c561190f6a584436a91b79b76c1495e3 2023-02-11 03:55:01,204 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-02-11 03:55:01,216 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-11 03:55:01,217 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:51376, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-11 03:55:01,224 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-02-11 03:55:01,225 INFO [RS-EventLoopGroup-1-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:42016, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-02-11 03:55:01,235 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151228/family1/dff6cba79d7e4606bba6d55affd63b26 first=Optional[r3] last=Optional[r6] 2023-02-11 03:55:01,236 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676087689083.614a4b4b2b88efbc57d9d64c1044323f., hostname=jenkins-hbase4.apache.org,37011,1676087618428, seqNum=2 for row with hfile group [{family1,hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151228/family1/dff6cba79d7e4606bba6d55affd63b26}] 2023-02-11 03:55:01,240 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.HStore(763): Validating hfile at hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151228/family1/dff6cba79d7e4606bba6d55affd63b26 for inclusion in 614a4b4b2b88efbc57d9d64c1044323f/family1 2023-02-11 03:55:01,242 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.HStore(775): HFile bounds: first=r3 last=r6 2023-02-11 03:55:01,242 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.HStore(777): Region bounds: first= last= 2023-02-11 03:55:01,242 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.HRegion(2445): Flush status journal for 614a4b4b2b88efbc57d9d64c1044323f: 2023-02-11 03:55:01,243 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151228/family1/dff6cba79d7e4606bba6d55affd63b26 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__b0f7u9ca82pnjn8ub938f202ogmm0o4f6sloro7ospvlj9thpkdbl6i7gf1j8hr5/family1/dff6cba79d7e4606bba6d55affd63b26 2023-02-11 03:55:01,245 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__b0f7u9ca82pnjn8ub938f202ogmm0o4f6sloro7ospvlj9thpkdbl6i7gf1j8hr5/family1/dff6cba79d7e4606bba6d55affd63b26 as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/614a4b4b2b88efbc57d9d64c1044323f/family1/bc631a3d2a924b2182d8da29f36c242a_SeqId_44_ 2023-02-11 03:55:01,245 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.HStore(849): Loaded HFile hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__b0f7u9ca82pnjn8ub938f202ogmm0o4f6sloro7ospvlj9thpkdbl6i7gf1j8hr5/family1/dff6cba79d7e4606bba6d55affd63b26 into 614a4b4b2b88efbc57d9d64c1044323f/family1 as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/614a4b4b2b88efbc57d9d64c1044323f/family1/bc631a3d2a924b2182d8da29f36c242a_SeqId_44_ - updating store file list. 2023-02-11 03:55:01,248 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.HStore(882): Loaded HFile hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/614a4b4b2b88efbc57d9d64c1044323f/family1/bc631a3d2a924b2182d8da29f36c242a_SeqId_44_ into 614a4b4b2b88efbc57d9d64c1044323f/family1 2023-02-11 03:55:01,249 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.HStore(855): Successfully loaded hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__b0f7u9ca82pnjn8ub938f202ogmm0o4f6sloro7ospvlj9thpkdbl6i7gf1j8hr5/family1/dff6cba79d7e4606bba6d55affd63b26 into 614a4b4b2b88efbc57d9d64c1044323f/family1 (new location: hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/614a4b4b2b88efbc57d9d64c1044323f/family1/bc631a3d2a924b2182d8da29f36c242a_SeqId_44_) 2023-02-11 03:55:01,249 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__b0f7u9ca82pnjn8ub938f202ogmm0o4f6sloro7ospvlj9thpkdbl6i7gf1j8hr5/family1/dff6cba79d7e4606bba6d55affd63b26 2023-02-11 03:55:01,249 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] compactions.SortedCompactionPolicy(75): Selecting compaction from 11 store files, 0 compacting, 11 eligible, 16 blocking 2023-02-11 03:55:01,252 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 10 files of size 51552 starting at candidate #1 after considering 44 permutations with 44 in ratio 2023-02-11 03:55:01,252 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.HStore(1912): 614a4b4b2b88efbc57d9d64c1044323f/family1 is initiating minor compaction 2023-02-11 03:55:01,252 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@3fbcaa53; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-11 03:55:01,252 INFO [RS:0;jenkins-hbase4:37011-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 614a4b4b2b88efbc57d9d64c1044323f/family1 in testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676087689083.614a4b4b2b88efbc57d9d64c1044323f. 2023-02-11 03:55:01,252 INFO [RS:0;jenkins-hbase4:37011-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/614a4b4b2b88efbc57d9d64c1044323f/family1/969d83483a884b32acfaccab43ab4139, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/614a4b4b2b88efbc57d9d64c1044323f/family1/601d6562a72a48aaa7d328507e2e4a94, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/614a4b4b2b88efbc57d9d64c1044323f/family1/7d5fe952ece94c4cbcf6e567be0c1641, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/614a4b4b2b88efbc57d9d64c1044323f/family1/ae5281d8d9db44ff810fe3418c870289, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/614a4b4b2b88efbc57d9d64c1044323f/family1/2b3a885b0f5649b4a13ce1e8c2482c4f, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/614a4b4b2b88efbc57d9d64c1044323f/family1/af123cf8b5d04e91b17b9e21cbbddcfd, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/614a4b4b2b88efbc57d9d64c1044323f/family1/30936f2ca08942579f4e4582ffd95218, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/614a4b4b2b88efbc57d9d64c1044323f/family1/d2c3624580d544e2a50bfa271c54f9aa, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/614a4b4b2b88efbc57d9d64c1044323f/family1/a0a87891a00b48ad87a274cb6061f4fc, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/614a4b4b2b88efbc57d9d64c1044323f/family1/bc631a3d2a924b2182d8da29f36c242a_SeqId_44_] into tmpdir=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/614a4b4b2b88efbc57d9d64c1044323f/.tmp, totalSize=50.3 K 2023-02-11 03:55:01,252 DEBUG [RS:0;jenkins-hbase4:37011-shortCompactions-0] compactions.Compactor(207): Compacting 969d83483a884b32acfaccab43ab4139, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=9 2023-02-11 03:55:01,252 DEBUG [RS:0;jenkins-hbase4:37011-shortCompactions-0] compactions.Compactor(207): Compacting 601d6562a72a48aaa7d328507e2e4a94, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=13 2023-02-11 03:55:01,252 DEBUG [RS:0;jenkins-hbase4:37011-shortCompactions-0] compactions.Compactor(207): Compacting 7d5fe952ece94c4cbcf6e567be0c1641, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=17 2023-02-11 03:55:01,253 DEBUG [RS:0;jenkins-hbase4:37011-shortCompactions-0] compactions.Compactor(207): Compacting ae5281d8d9db44ff810fe3418c870289, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=21 2023-02-11 03:55:01,253 DEBUG [RS:0;jenkins-hbase4:37011-shortCompactions-0] compactions.Compactor(207): Compacting 2b3a885b0f5649b4a13ce1e8c2482c4f, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=25 2023-02-11 03:55:01,253 DEBUG [RS:0;jenkins-hbase4:37011-shortCompactions-0] compactions.Compactor(207): Compacting af123cf8b5d04e91b17b9e21cbbddcfd, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=29 2023-02-11 03:55:01,253 DEBUG [RS:0;jenkins-hbase4:37011-shortCompactions-0] compactions.Compactor(207): Compacting 30936f2ca08942579f4e4582ffd95218, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33 2023-02-11 03:55:01,253 DEBUG [RS:0;jenkins-hbase4:37011-shortCompactions-0] compactions.Compactor(207): Compacting d2c3624580d544e2a50bfa271c54f9aa, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=37 2023-02-11 03:55:01,253 DEBUG [RS:0;jenkins-hbase4:37011-shortCompactions-0] compactions.Compactor(207): Compacting a0a87891a00b48ad87a274cb6061f4fc, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=41 2023-02-11 03:55:01,253 DEBUG [RS:0;jenkins-hbase4:37011-shortCompactions-0] compactions.Compactor(207): Compacting bc631a3d2a924b2182d8da29f36c242a_SeqId_44_, keycount=4, bloomtype=ROW, size=5.6 K, encoding=NONE, compression=NONE, seqNum=45 2023-02-11 03:55:01,254 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-02-11 03:55:01,255 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-11 03:55:01,256 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151228d7eee638ea8b45578845c99c799736a1 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151228d7eee638ea8b45578845c99c799736a1 2023-02-11 03:55:01,257 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201512315446b1245496496ab72413015d087f07 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201512315446b1245496496ab72413015d087f07 2023-02-11 03:55:01,258 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201601014c4a5b1ab9c74800b0c69eb1234e8bb9 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201601014c4a5b1ab9c74800b0c69eb1234e8bb9 2023-02-11 03:55:01,260 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201601037098c7fe2db6417eb0fbb457d674a234 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201601037098c7fe2db6417eb0fbb457d674a234 2023-02-11 03:55:01,260 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 4 expired mob files are deleted 2023-02-11 03:55:01,260 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151228' 2023-02-11 03:55:01,261 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-11 03:55:01,261 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-11 03:55:01,261 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-11 03:55:01,261 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-11 03:55:01,261 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-11 03:55:01,261 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-11 03:55:01,261 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-11 03:55:01,261 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-11 03:55:01,294 INFO [RS:0;jenkins-hbase4:37011-shortCompactions-0] throttle.PressureAwareThroughputController(145): 614a4b4b2b88efbc57d9d64c1044323f#family1#compaction#72 average throughput is 0.79 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 61.54 MB/second 2023-02-11 03:55:01,727 DEBUG [RS:0;jenkins-hbase4:37011-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/614a4b4b2b88efbc57d9d64c1044323f/.tmp/family1/2b37d2ef7114456f87cd060524ba65ad as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/614a4b4b2b88efbc57d9d64c1044323f/family1/2b37d2ef7114456f87cd060524ba65ad 2023-02-11 03:55:01,731 INFO [RS:0;jenkins-hbase4:37011-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 10 file(s) in 614a4b4b2b88efbc57d9d64c1044323f/family1 of 614a4b4b2b88efbc57d9d64c1044323f into 2b37d2ef7114456f87cd060524ba65ad(size=6.8 K), total size for store is 197.2 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-11 03:55:01,732 DEBUG [RS:0;jenkins-hbase4:37011-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 614a4b4b2b88efbc57d9d64c1044323f: 2023-02-11 03:55:01,732 INFO [RS:0;jenkins-hbase4:37011-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676087689083.614a4b4b2b88efbc57d9d64c1044323f., storeName=family1, fileCount=10, fileSize=50.3 K (5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.6 K), priority=2, time=1676087701252; duration=0sec 2023-02-11 03:55:01,732 DEBUG [RS:0;jenkins-hbase4:37011-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-11 03:55:01,801 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/614a4b4b2b88efbc57d9d64c1044323f/family1/969d83483a884b32acfaccab43ab4139, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/614a4b4b2b88efbc57d9d64c1044323f/family1/601d6562a72a48aaa7d328507e2e4a94, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/614a4b4b2b88efbc57d9d64c1044323f/family1/7d5fe952ece94c4cbcf6e567be0c1641, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/614a4b4b2b88efbc57d9d64c1044323f/family1/ae5281d8d9db44ff810fe3418c870289, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/614a4b4b2b88efbc57d9d64c1044323f/family1/2b3a885b0f5649b4a13ce1e8c2482c4f, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/614a4b4b2b88efbc57d9d64c1044323f/family1/af123cf8b5d04e91b17b9e21cbbddcfd, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/614a4b4b2b88efbc57d9d64c1044323f/family1/30936f2ca08942579f4e4582ffd95218, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/614a4b4b2b88efbc57d9d64c1044323f/family1/d2c3624580d544e2a50bfa271c54f9aa, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/614a4b4b2b88efbc57d9d64c1044323f/family1/a0a87891a00b48ad87a274cb6061f4fc, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/614a4b4b2b88efbc57d9d64c1044323f/family1/bc631a3d2a924b2182d8da29f36c242a_SeqId_44_] to archive 2023-02-11 03:55:01,802 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-11 03:55:01,804 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/614a4b4b2b88efbc57d9d64c1044323f/family1/969d83483a884b32acfaccab43ab4139 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/614a4b4b2b88efbc57d9d64c1044323f/family1/969d83483a884b32acfaccab43ab4139 2023-02-11 03:55:01,805 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/614a4b4b2b88efbc57d9d64c1044323f/family1/601d6562a72a48aaa7d328507e2e4a94 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/614a4b4b2b88efbc57d9d64c1044323f/family1/601d6562a72a48aaa7d328507e2e4a94 2023-02-11 03:55:01,806 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/614a4b4b2b88efbc57d9d64c1044323f/family1/7d5fe952ece94c4cbcf6e567be0c1641 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/614a4b4b2b88efbc57d9d64c1044323f/family1/7d5fe952ece94c4cbcf6e567be0c1641 2023-02-11 03:55:01,807 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/614a4b4b2b88efbc57d9d64c1044323f/family1/ae5281d8d9db44ff810fe3418c870289 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/614a4b4b2b88efbc57d9d64c1044323f/family1/ae5281d8d9db44ff810fe3418c870289 2023-02-11 03:55:01,809 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/614a4b4b2b88efbc57d9d64c1044323f/family1/2b3a885b0f5649b4a13ce1e8c2482c4f to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/614a4b4b2b88efbc57d9d64c1044323f/family1/2b3a885b0f5649b4a13ce1e8c2482c4f 2023-02-11 03:55:01,810 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/614a4b4b2b88efbc57d9d64c1044323f/family1/af123cf8b5d04e91b17b9e21cbbddcfd to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/614a4b4b2b88efbc57d9d64c1044323f/family1/af123cf8b5d04e91b17b9e21cbbddcfd 2023-02-11 03:55:01,811 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/614a4b4b2b88efbc57d9d64c1044323f/family1/30936f2ca08942579f4e4582ffd95218 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/614a4b4b2b88efbc57d9d64c1044323f/family1/30936f2ca08942579f4e4582ffd95218 2023-02-11 03:55:01,812 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/614a4b4b2b88efbc57d9d64c1044323f/family1/d2c3624580d544e2a50bfa271c54f9aa to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/614a4b4b2b88efbc57d9d64c1044323f/family1/d2c3624580d544e2a50bfa271c54f9aa 2023-02-11 03:55:01,813 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/614a4b4b2b88efbc57d9d64c1044323f/family1/a0a87891a00b48ad87a274cb6061f4fc to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/614a4b4b2b88efbc57d9d64c1044323f/family1/a0a87891a00b48ad87a274cb6061f4fc 2023-02-11 03:55:01,814 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/614a4b4b2b88efbc57d9d64c1044323f/family1/bc631a3d2a924b2182d8da29f36c242a_SeqId_44_ to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/614a4b4b2b88efbc57d9d64c1044323f/family1/bc631a3d2a924b2182d8da29f36c242a_SeqId_44_ 2023-02-11 03:55:02,125 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151205470a161cf10d4bcfa462fdd9db7fa753 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151205470a161cf10d4bcfa462fdd9db7fa753 2023-02-11 03:55:02,126 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-02-11 03:55:02,148 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151130/family1/48909c39e055409a80c0dce3c300d300 first=Optional[r00] last=Optional[r2] 2023-02-11 03:55:02,149 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676087689083.614a4b4b2b88efbc57d9d64c1044323f., hostname=jenkins-hbase4.apache.org,37011,1676087618428, seqNum=2 for row with hfile group [{family1,hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151130/family1/48909c39e055409a80c0dce3c300d300}] 2023-02-11 03:55:02,150 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.HStore(763): Validating hfile at hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151130/family1/48909c39e055409a80c0dce3c300d300 for inclusion in 614a4b4b2b88efbc57d9d64c1044323f/family1 2023-02-11 03:55:02,154 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.HStore(775): HFile bounds: first=r00 last=r2 2023-02-11 03:55:02,154 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.HStore(777): Region bounds: first= last= 2023-02-11 03:55:02,154 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.HRegion(2445): Flush status journal for 614a4b4b2b88efbc57d9d64c1044323f: 2023-02-11 03:55:02,155 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151130/family1/48909c39e055409a80c0dce3c300d300 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__u73rcfu8v0n3gqfq12ok4hhb357uh679a11dqvol2vp1hdr0ieelolcr7vuvdigo/family1/48909c39e055409a80c0dce3c300d300 2023-02-11 03:55:02,157 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__u73rcfu8v0n3gqfq12ok4hhb357uh679a11dqvol2vp1hdr0ieelolcr7vuvdigo/family1/48909c39e055409a80c0dce3c300d300 as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/614a4b4b2b88efbc57d9d64c1044323f/family1/c2f6ac5ec0bf41bdad9541950a9ccf79_SeqId_47_ 2023-02-11 03:55:02,157 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.HStore(849): Loaded HFile hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__u73rcfu8v0n3gqfq12ok4hhb357uh679a11dqvol2vp1hdr0ieelolcr7vuvdigo/family1/48909c39e055409a80c0dce3c300d300 into 614a4b4b2b88efbc57d9d64c1044323f/family1 as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/614a4b4b2b88efbc57d9d64c1044323f/family1/c2f6ac5ec0bf41bdad9541950a9ccf79_SeqId_47_ - updating store file list. 2023-02-11 03:55:02,161 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.HStore(882): Loaded HFile hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/614a4b4b2b88efbc57d9d64c1044323f/family1/c2f6ac5ec0bf41bdad9541950a9ccf79_SeqId_47_ into 614a4b4b2b88efbc57d9d64c1044323f/family1 2023-02-11 03:55:02,161 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.HStore(855): Successfully loaded hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__u73rcfu8v0n3gqfq12ok4hhb357uh679a11dqvol2vp1hdr0ieelolcr7vuvdigo/family1/48909c39e055409a80c0dce3c300d300 into 614a4b4b2b88efbc57d9d64c1044323f/family1 (new location: hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/614a4b4b2b88efbc57d9d64c1044323f/family1/c2f6ac5ec0bf41bdad9541950a9ccf79_SeqId_47_) 2023-02-11 03:55:02,161 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__u73rcfu8v0n3gqfq12ok4hhb357uh679a11dqvol2vp1hdr0ieelolcr7vuvdigo/family1/48909c39e055409a80c0dce3c300d300 2023-02-11 03:55:02,162 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-02-11 03:55:02,162 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 397422 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-02-11 03:55:02,163 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.HStore(1912): 614a4b4b2b88efbc57d9d64c1044323f/family1 is initiating minor compaction (all files) 2023-02-11 03:55:02,163 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@4fda489; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-11 03:55:02,163 INFO [RS:0;jenkins-hbase4:37011-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 614a4b4b2b88efbc57d9d64c1044323f/family1 in testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676087689083.614a4b4b2b88efbc57d9d64c1044323f. 2023-02-11 03:55:02,163 INFO [RS:0;jenkins-hbase4:37011-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/614a4b4b2b88efbc57d9d64c1044323f/family1/9824724851a8448685c3cdd9e40b73ab, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/614a4b4b2b88efbc57d9d64c1044323f/family1/2b37d2ef7114456f87cd060524ba65ad, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/614a4b4b2b88efbc57d9d64c1044323f/family1/c2f6ac5ec0bf41bdad9541950a9ccf79_SeqId_47_] into tmpdir=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/614a4b4b2b88efbc57d9d64c1044323f/.tmp, totalSize=388.1 K 2023-02-11 03:55:02,163 DEBUG [RS:0;jenkins-hbase4:37011-shortCompactions-0] compactions.Compactor(207): Compacting 9824724851a8448685c3cdd9e40b73ab, keycount=1000, bloomtype=ROW, size=190.5 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1448878842014 2023-02-11 03:55:02,164 DEBUG [RS:0;jenkins-hbase4:37011-shortCompactions-0] compactions.Compactor(207): Compacting 2b37d2ef7114456f87cd060524ba65ad, keycount=9, bloomtype=ROW, size=6.8 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1441621242014 2023-02-11 03:55:02,164 DEBUG [RS:0;jenkins-hbase4:37011-shortCompactions-0] compactions.Compactor(207): Compacting c2f6ac5ec0bf41bdad9541950a9ccf79_SeqId_47_, keycount=1002, bloomtype=ROW, size=190.9 K, encoding=NONE, compression=NONE, seqNum=48, earliestPutTs=1448878842014 2023-02-11 03:55:02,164 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-02-11 03:55:02,165 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-11 03:55:02,166 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151130984a7d1aeb464af0b2519dba3ccaf447 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151130984a7d1aeb464af0b2519dba3ccaf447 2023-02-11 03:55:02,167 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151201e3cdfa2e6ec047b584c1eefd9398d097 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151201e3cdfa2e6ec047b584c1eefd9398d097 2023-02-11 03:55:02,169 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e2015120536f6952537dc4b37bb6e12cd74a2cf6f to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e2015120536f6952537dc4b37bb6e12cd74a2cf6f 2023-02-11 03:55:02,169 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 3 expired mob files are deleted 2023-02-11 03:55:02,169 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151130' 2023-02-11 03:55:02,169 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 3 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-11 03:55:02,169 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676087621935] compactions.PartitionedMobCompactor(354): After compaction, there are 5 mob files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-02-11 03:55:02,169 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676087621935] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-11 03:55:02,170 DEBUG [PEWorker-2] locking.LockProcedure(242): UNLOCKED pid=58, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-02-11 03:55:02,172 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=58, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE in 1.8090 sec 2023-02-11 03:55:02,180 INFO [RS:0;jenkins-hbase4:37011-shortCompactions-0] throttle.PressureAwareThroughputController(145): 614a4b4b2b88efbc57d9d64c1044323f#family1#compaction#73 average throughput is 22.45 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 61.54 MB/second 2023-02-11 03:55:02,246 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] master.HMaster$12(2441): Client=jenkins//172.31.14.131 modify table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily from 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} to 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'} 2023-02-11 03:55:02,247 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] procedure2.ProcedureExecutor(1029): Stored pid=59, state=RUNNABLE:MODIFY_TABLE_PREPARE; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:55:02,656 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/.tabledesc/.tableinfo.0000000002 2023-02-11 03:55:02,657 DEBUG [PEWorker-1] util.FSTableDescriptors(519): Deleted hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/.tabledesc/.tableinfo.0000000001 2023-02-11 03:55:02,657 INFO [PEWorker-1] util.FSTableDescriptors(293): Updated tableinfo=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/.tabledesc/.tableinfo.0000000002 2023-02-11 03:55:02,659 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=60, ppid=59, state=RUNNABLE:REOPEN_TABLE_REGIONS_GET_REGIONS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily}] 2023-02-11 03:55:02,661 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=61, ppid=60, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=614a4b4b2b88efbc57d9d64c1044323f, REOPEN/MOVE}] 2023-02-11 03:55:02,662 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=61, ppid=60, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=614a4b4b2b88efbc57d9d64c1044323f, REOPEN/MOVE 2023-02-11 03:55:02,663 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=61 updating hbase:meta row=614a4b4b2b88efbc57d9d64c1044323f, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:55:02,663 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676087689083.614a4b4b2b88efbc57d9d64c1044323f.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1676087702662"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676087702662"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676087702662"}]},"ts":"1676087702662"} 2023-02-11 03:55:02,664 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=62, ppid=61, state=RUNNABLE; CloseRegionProcedure 614a4b4b2b88efbc57d9d64c1044323f, server=jenkins-hbase4.apache.org,37011,1676087618428}] 2023-02-11 03:55:02,815 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 614a4b4b2b88efbc57d9d64c1044323f 2023-02-11 03:55:02,816 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 614a4b4b2b88efbc57d9d64c1044323f, disabling compactions & flushes 2023-02-11 03:55:02,816 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1857): waiting for 1 compactions to complete for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676087689083.614a4b4b2b88efbc57d9d64c1044323f. 2023-02-11 03:55:03,396 DEBUG [RS:0;jenkins-hbase4:37011-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/614a4b4b2b88efbc57d9d64c1044323f/.tmp/family1/15da5851d7cc4e298e8f6ae30dec3114 as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/614a4b4b2b88efbc57d9d64c1044323f/family1/15da5851d7cc4e298e8f6ae30dec3114 2023-02-11 03:55:03,401 INFO [RS:0;jenkins-hbase4:37011-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in 614a4b4b2b88efbc57d9d64c1044323f/family1 of 614a4b4b2b88efbc57d9d64c1044323f into 15da5851d7cc4e298e8f6ae30dec3114(size=192.0 K), total size for store is 192.0 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-02-11 03:55:03,401 DEBUG [RS:0;jenkins-hbase4:37011-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 614a4b4b2b88efbc57d9d64c1044323f: 2023-02-11 03:55:03,401 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676087689083.614a4b4b2b88efbc57d9d64c1044323f. 2023-02-11 03:55:03,401 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676087689083.614a4b4b2b88efbc57d9d64c1044323f. 2023-02-11 03:55:03,401 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676087689083.614a4b4b2b88efbc57d9d64c1044323f. after waiting 0 ms 2023-02-11 03:55:03,401 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676087689083.614a4b4b2b88efbc57d9d64c1044323f. 2023-02-11 03:55:03,401 INFO [RS:0;jenkins-hbase4:37011-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676087689083.614a4b4b2b88efbc57d9d64c1044323f., storeName=family1, fileCount=3, fileSize=388.1 K (190.5 K, 6.8 K, 190.9 K), priority=2, time=1676087702163; duration=1sec 2023-02-11 03:55:03,401 DEBUG [RS:0;jenkins-hbase4:37011-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-11 03:55:03,404 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676087689083.614a4b4b2b88efbc57d9d64c1044323f.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/614a4b4b2b88efbc57d9d64c1044323f/family1/9824724851a8448685c3cdd9e40b73ab, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/614a4b4b2b88efbc57d9d64c1044323f/family1/2b37d2ef7114456f87cd060524ba65ad, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/614a4b4b2b88efbc57d9d64c1044323f/family1/c2f6ac5ec0bf41bdad9541950a9ccf79_SeqId_47_] to archive 2023-02-11 03:55:03,405 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676087689083.614a4b4b2b88efbc57d9d64c1044323f.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-11 03:55:03,406 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676087689083.614a4b4b2b88efbc57d9d64c1044323f.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/614a4b4b2b88efbc57d9d64c1044323f/family1/9824724851a8448685c3cdd9e40b73ab to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/614a4b4b2b88efbc57d9d64c1044323f/family1/9824724851a8448685c3cdd9e40b73ab 2023-02-11 03:55:03,407 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676087689083.614a4b4b2b88efbc57d9d64c1044323f.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/614a4b4b2b88efbc57d9d64c1044323f/family1/2b37d2ef7114456f87cd060524ba65ad to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/614a4b4b2b88efbc57d9d64c1044323f/family1/2b37d2ef7114456f87cd060524ba65ad 2023-02-11 03:55:03,408 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676087689083.614a4b4b2b88efbc57d9d64c1044323f.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/614a4b4b2b88efbc57d9d64c1044323f/family1/c2f6ac5ec0bf41bdad9541950a9ccf79_SeqId_47_ to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/614a4b4b2b88efbc57d9d64c1044323f/family1/c2f6ac5ec0bf41bdad9541950a9ccf79_SeqId_47_ 2023-02-11 03:55:03,413 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/614a4b4b2b88efbc57d9d64c1044323f/recovered.edits/50.seqid, newMaxSeqId=50, maxSeqId=1 2023-02-11 03:55:03,413 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676087689083.614a4b4b2b88efbc57d9d64c1044323f. 2023-02-11 03:55:03,413 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 614a4b4b2b88efbc57d9d64c1044323f: Waiting for close lock at 1676087702816Running coprocessor pre-close hooks at 1676087702816Disabling compacts and flushes for region at 1676087702816Disabling writes for close at 1676087703401 (+585 ms)Writing region close event to WAL at 1676087703410 (+9 ms)Running coprocessor post-close hooks at 1676087703413 (+3 ms)Closed at 1676087703413 2023-02-11 03:55:03,413 WARN [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(3509): Not adding moved region record: 614a4b4b2b88efbc57d9d64c1044323f to self. 2023-02-11 03:55:03,415 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 614a4b4b2b88efbc57d9d64c1044323f 2023-02-11 03:55:03,415 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=61 updating hbase:meta row=614a4b4b2b88efbc57d9d64c1044323f, regionState=CLOSED 2023-02-11 03:55:03,415 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676087689083.614a4b4b2b88efbc57d9d64c1044323f.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1676087703415"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676087703415"}]},"ts":"1676087703415"} 2023-02-11 03:55:03,417 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=62, resume processing ppid=61 2023-02-11 03:55:03,417 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=62, ppid=61, state=SUCCESS; CloseRegionProcedure 614a4b4b2b88efbc57d9d64c1044323f, server=jenkins-hbase4.apache.org,37011,1676087618428 in 752 msec 2023-02-11 03:55:03,418 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=61, ppid=60, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=614a4b4b2b88efbc57d9d64c1044323f, REOPEN/MOVE; state=CLOSED, location=jenkins-hbase4.apache.org,37011,1676087618428; forceNewPlan=false, retain=true 2023-02-11 03:55:03,568 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=61 updating hbase:meta row=614a4b4b2b88efbc57d9d64c1044323f, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:55:03,568 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676087689083.614a4b4b2b88efbc57d9d64c1044323f.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1676087703568"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676087703568"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676087703568"}]},"ts":"1676087703568"} 2023-02-11 03:55:03,570 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=63, ppid=61, state=RUNNABLE; OpenRegionProcedure 614a4b4b2b88efbc57d9d64c1044323f, server=jenkins-hbase4.apache.org,37011,1676087618428}] 2023-02-11 03:55:03,723 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676087689083.614a4b4b2b88efbc57d9d64c1044323f. 2023-02-11 03:55:03,724 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 614a4b4b2b88efbc57d9d64c1044323f, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676087689083.614a4b4b2b88efbc57d9d64c1044323f.', STARTKEY => '', ENDKEY => ''} 2023-02-11 03:55:03,724 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 614a4b4b2b88efbc57d9d64c1044323f 2023-02-11 03:55:03,724 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676087689083.614a4b4b2b88efbc57d9d64c1044323f.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-11 03:55:03,724 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 614a4b4b2b88efbc57d9d64c1044323f 2023-02-11 03:55:03,724 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 614a4b4b2b88efbc57d9d64c1044323f 2023-02-11 03:55:03,725 INFO [StoreOpener-614a4b4b2b88efbc57d9d64c1044323f-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 614a4b4b2b88efbc57d9d64c1044323f 2023-02-11 03:55:03,726 DEBUG [StoreOpener-614a4b4b2b88efbc57d9d64c1044323f-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/614a4b4b2b88efbc57d9d64c1044323f/family1 2023-02-11 03:55:03,726 DEBUG [StoreOpener-614a4b4b2b88efbc57d9d64c1044323f-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/614a4b4b2b88efbc57d9d64c1044323f/family1 2023-02-11 03:55:03,726 INFO [StoreOpener-614a4b4b2b88efbc57d9d64c1044323f-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 614a4b4b2b88efbc57d9d64c1044323f columnFamilyName family1 2023-02-11 03:55:03,731 DEBUG [StoreOpener-614a4b4b2b88efbc57d9d64c1044323f-1] regionserver.HStore(539): loaded hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/614a4b4b2b88efbc57d9d64c1044323f/family1/15da5851d7cc4e298e8f6ae30dec3114 2023-02-11 03:55:03,731 INFO [StoreOpener-614a4b4b2b88efbc57d9d64c1044323f-1] regionserver.HStore(310): Store=614a4b4b2b88efbc57d9d64c1044323f/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 03:55:03,732 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/614a4b4b2b88efbc57d9d64c1044323f 2023-02-11 03:55:03,732 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/614a4b4b2b88efbc57d9d64c1044323f 2023-02-11 03:55:03,735 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 614a4b4b2b88efbc57d9d64c1044323f 2023-02-11 03:55:03,735 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 614a4b4b2b88efbc57d9d64c1044323f; next sequenceid=51; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=68273360, jitterRate=0.01735234260559082}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-11 03:55:03,736 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 614a4b4b2b88efbc57d9d64c1044323f: Running coprocessor pre-open hook at 1676087703724Writing region info on filesystem at 1676087703724Initializing all the Stores at 1676087703725 (+1 ms)Instantiating store for column family {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'} at 1676087703725Cleaning up temporary data from old regions at 1676087703733 (+8 ms)Cleaning up detritus from prior splits at 1676087703733Running coprocessor post-open hooks at 1676087703735 (+2 ms)Region opened successfully at 1676087703735 2023-02-11 03:55:03,736 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676087689083.614a4b4b2b88efbc57d9d64c1044323f., pid=63, masterSystemTime=1676087703721 2023-02-11 03:55:03,737 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676087689083.614a4b4b2b88efbc57d9d64c1044323f. 2023-02-11 03:55:03,737 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676087689083.614a4b4b2b88efbc57d9d64c1044323f. 2023-02-11 03:55:03,738 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=61 updating hbase:meta row=614a4b4b2b88efbc57d9d64c1044323f, regionState=OPEN, openSeqNum=51, regionLocation=jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:55:03,738 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676087689083.614a4b4b2b88efbc57d9d64c1044323f.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1676087703738"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676087703738"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676087703738"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676087703738"}]},"ts":"1676087703738"} 2023-02-11 03:55:03,740 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=63, resume processing ppid=61 2023-02-11 03:55:03,740 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=63, ppid=61, state=SUCCESS; OpenRegionProcedure 614a4b4b2b88efbc57d9d64c1044323f, server=jenkins-hbase4.apache.org,37011,1676087618428 in 170 msec 2023-02-11 03:55:03,741 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=61, resume processing ppid=60 2023-02-11 03:55:03,741 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=61, ppid=60, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=614a4b4b2b88efbc57d9d64c1044323f, REOPEN/MOVE in 1.0790 sec 2023-02-11 03:55:03,743 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=60, resume processing ppid=59 2023-02-11 03:55:03,743 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=60, ppid=59, state=SUCCESS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily in 1.0830 sec 2023-02-11 03:55:03,745 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=59, state=SUCCESS; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily in 1.4970 sec 2023-02-11 03:55:03,746 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] master.MasterRpcServices(1227): Checking to see if procedure is done pid=59 2023-02-11 03:55:03,747 INFO [Listener at localhost/46205] compactions.TestMobCompactor(278): alter status finished 2023-02-11 03:55:03,748 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=39921] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'}] of the table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:55:03,749 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676087621935] procedure2.ProcedureExecutor(1029): Stored pid=64, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-02-11 03:55:03,750 DEBUG [PEWorker-3] locking.LockProcedure(309): LOCKED pid=64, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-02-11 03:55:03,750 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676087621935] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-11 03:55:03,750 INFO [PEWorker-3] procedure2.TimeoutExecutorThread(81): ADDED pid=64, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1676088303750 2023-02-11 03:55:03,751 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676087621935] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-02-11 03:55:03,751 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676087621935] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 5 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column 'family1' 2023-02-11 03:55:03,751 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676087621935] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-02-11 03:55:03,752 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676087621935] zookeeper.ReadOnlyZKClient(139): Connect 0x6be34b78 to 127.0.0.1:53383 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-11 03:55:03,756 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676087621935] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@49713462, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-11 03:55:03,756 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-11 03:55:03,756 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-11 03:55:03,756 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-11 03:55:03,756 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-11 03:55:03,756 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-11 03:55:04,579 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151128f084008e8dd14804a308ccf8d71324fc to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151128f084008e8dd14804a308ccf8d71324fc 2023-02-11 03:55:04,580 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-02-11 03:55:04,596 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-11 03:55:04,597 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:36730, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-11 03:55:04,603 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-02-11 03:55:04,604 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:43526, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-02-11 03:55:04,614 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151101/family1/deeffbcdefc34f58a6c5792ca098a795 first=Optional[r05] last=Optional[r8] 2023-02-11 03:55:04,614 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676087689083.614a4b4b2b88efbc57d9d64c1044323f., hostname=jenkins-hbase4.apache.org,37011,1676087618428, seqNum=51 for row with hfile group [{family1,hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151101/family1/deeffbcdefc34f58a6c5792ca098a795}] 2023-02-11 03:55:04,618 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.HStore(763): Validating hfile at hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151101/family1/deeffbcdefc34f58a6c5792ca098a795 for inclusion in 614a4b4b2b88efbc57d9d64c1044323f/family1 2023-02-11 03:55:04,620 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.HStore(775): HFile bounds: first=r05 last=r8 2023-02-11 03:55:04,620 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.HStore(777): Region bounds: first= last= 2023-02-11 03:55:04,620 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.HRegion(2445): Flush status journal for 614a4b4b2b88efbc57d9d64c1044323f: 2023-02-11 03:55:04,621 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151101/family1/deeffbcdefc34f58a6c5792ca098a795 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__gjpf6211579d6tgr7bmij0r6qim60i1c9lljiilbjf3m3fjiisp3ar3c0qqini85/family1/deeffbcdefc34f58a6c5792ca098a795 2023-02-11 03:55:04,623 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__gjpf6211579d6tgr7bmij0r6qim60i1c9lljiilbjf3m3fjiisp3ar3c0qqini85/family1/deeffbcdefc34f58a6c5792ca098a795 as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/614a4b4b2b88efbc57d9d64c1044323f/family1/ddc1531319754c3fb49047964998ab6c_SeqId_53_ 2023-02-11 03:55:04,623 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.HStore(849): Loaded HFile hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__gjpf6211579d6tgr7bmij0r6qim60i1c9lljiilbjf3m3fjiisp3ar3c0qqini85/family1/deeffbcdefc34f58a6c5792ca098a795 into 614a4b4b2b88efbc57d9d64c1044323f/family1 as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/614a4b4b2b88efbc57d9d64c1044323f/family1/ddc1531319754c3fb49047964998ab6c_SeqId_53_ - updating store file list. 2023-02-11 03:55:04,627 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.HStore(882): Loaded HFile hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/614a4b4b2b88efbc57d9d64c1044323f/family1/ddc1531319754c3fb49047964998ab6c_SeqId_53_ into 614a4b4b2b88efbc57d9d64c1044323f/family1 2023-02-11 03:55:04,627 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.HStore(855): Successfully loaded hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__gjpf6211579d6tgr7bmij0r6qim60i1c9lljiilbjf3m3fjiisp3ar3c0qqini85/family1/deeffbcdefc34f58a6c5792ca098a795 into 614a4b4b2b88efbc57d9d64c1044323f/family1 (new location: hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/614a4b4b2b88efbc57d9d64c1044323f/family1/ddc1531319754c3fb49047964998ab6c_SeqId_53_) 2023-02-11 03:55:04,627 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__gjpf6211579d6tgr7bmij0r6qim60i1c9lljiilbjf3m3fjiisp3ar3c0qqini85/family1/deeffbcdefc34f58a6c5792ca098a795 2023-02-11 03:55:04,628 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-02-11 03:55:04,629 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-11 03:55:04,630 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201511209f41ce1c7a524ab7ad116d13285b5fe1 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201511209f41ce1c7a524ab7ad116d13285b5fe1 2023-02-11 03:55:04,631 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151128b71e1b0d76d448978901f9dbb635fc87 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151128b71e1b0d76d448978901f9dbb635fc87 2023-02-11 03:55:04,631 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2 expired mob files are deleted 2023-02-11 03:55:04,631 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151101' 2023-02-11 03:55:04,632 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 2 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-11 03:55:04,632 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-11 03:55:04,632 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-11 03:55:04,632 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676087621935] compactions.PartitionedMobCompactor(354): After compaction, there are 4 mob files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-02-11 03:55:04,632 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676087621935] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-11 03:55:04,633 DEBUG [PEWorker-5] locking.LockProcedure(242): UNLOCKED pid=64, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-02-11 03:55:04,634 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=64, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE in 885 msec 2023-02-11 03:55:04,681 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] master.HMaster$12(2441): Client=jenkins//172.31.14.131 modify table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily from 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'} to 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'DAILY'} 2023-02-11 03:55:04,682 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] procedure2.ProcedureExecutor(1029): Stored pid=65, state=RUNNABLE:MODIFY_TABLE_PREPARE; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:55:05,090 DEBUG [PEWorker-2] util.FSTableDescriptors(570): Wrote into hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/.tabledesc/.tableinfo.0000000003 2023-02-11 03:55:05,091 DEBUG [PEWorker-2] util.FSTableDescriptors(519): Deleted hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/.tabledesc/.tableinfo.0000000002 2023-02-11 03:55:05,091 INFO [PEWorker-2] util.FSTableDescriptors(293): Updated tableinfo=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/.tabledesc/.tableinfo.0000000003 2023-02-11 03:55:05,093 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=66, ppid=65, state=RUNNABLE:REOPEN_TABLE_REGIONS_GET_REGIONS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily}] 2023-02-11 03:55:05,095 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=67, ppid=66, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=614a4b4b2b88efbc57d9d64c1044323f, REOPEN/MOVE}] 2023-02-11 03:55:05,096 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=67, ppid=66, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=614a4b4b2b88efbc57d9d64c1044323f, REOPEN/MOVE 2023-02-11 03:55:05,096 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=67 updating hbase:meta row=614a4b4b2b88efbc57d9d64c1044323f, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:55:05,096 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676087689083.614a4b4b2b88efbc57d9d64c1044323f.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1676087705096"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676087705096"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676087705096"}]},"ts":"1676087705096"} 2023-02-11 03:55:05,097 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=68, ppid=67, state=RUNNABLE; CloseRegionProcedure 614a4b4b2b88efbc57d9d64c1044323f, server=jenkins-hbase4.apache.org,37011,1676087618428}] 2023-02-11 03:55:05,249 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 614a4b4b2b88efbc57d9d64c1044323f 2023-02-11 03:55:05,249 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 614a4b4b2b88efbc57d9d64c1044323f, disabling compactions & flushes 2023-02-11 03:55:05,249 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676087689083.614a4b4b2b88efbc57d9d64c1044323f. 2023-02-11 03:55:05,249 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676087689083.614a4b4b2b88efbc57d9d64c1044323f. 2023-02-11 03:55:05,249 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676087689083.614a4b4b2b88efbc57d9d64c1044323f. after waiting 0 ms 2023-02-11 03:55:05,249 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676087689083.614a4b4b2b88efbc57d9d64c1044323f. 2023-02-11 03:55:05,256 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/614a4b4b2b88efbc57d9d64c1044323f/recovered.edits/55.seqid, newMaxSeqId=55, maxSeqId=50 2023-02-11 03:55:05,257 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676087689083.614a4b4b2b88efbc57d9d64c1044323f. 2023-02-11 03:55:05,257 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 614a4b4b2b88efbc57d9d64c1044323f: Waiting for close lock at 1676087705249Running coprocessor pre-close hooks at 1676087705249Disabling compacts and flushes for region at 1676087705249Disabling writes for close at 1676087705249Writing region close event to WAL at 1676087705253 (+4 ms)Running coprocessor post-close hooks at 1676087705257 (+4 ms)Closed at 1676087705257 2023-02-11 03:55:05,257 WARN [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(3509): Not adding moved region record: 614a4b4b2b88efbc57d9d64c1044323f to self. 2023-02-11 03:55:05,258 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 614a4b4b2b88efbc57d9d64c1044323f 2023-02-11 03:55:05,258 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=67 updating hbase:meta row=614a4b4b2b88efbc57d9d64c1044323f, regionState=CLOSED 2023-02-11 03:55:05,258 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676087689083.614a4b4b2b88efbc57d9d64c1044323f.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1676087705258"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676087705258"}]},"ts":"1676087705258"} 2023-02-11 03:55:05,260 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=68, resume processing ppid=67 2023-02-11 03:55:05,260 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=68, ppid=67, state=SUCCESS; CloseRegionProcedure 614a4b4b2b88efbc57d9d64c1044323f, server=jenkins-hbase4.apache.org,37011,1676087618428 in 162 msec 2023-02-11 03:55:05,261 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=67, ppid=66, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=614a4b4b2b88efbc57d9d64c1044323f, REOPEN/MOVE; state=CLOSED, location=jenkins-hbase4.apache.org,37011,1676087618428; forceNewPlan=false, retain=true 2023-02-11 03:55:05,411 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=67 updating hbase:meta row=614a4b4b2b88efbc57d9d64c1044323f, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:55:05,411 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676087689083.614a4b4b2b88efbc57d9d64c1044323f.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1676087705411"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676087705411"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676087705411"}]},"ts":"1676087705411"} 2023-02-11 03:55:05,413 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=69, ppid=67, state=RUNNABLE; OpenRegionProcedure 614a4b4b2b88efbc57d9d64c1044323f, server=jenkins-hbase4.apache.org,37011,1676087618428}] 2023-02-11 03:55:05,566 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676087689083.614a4b4b2b88efbc57d9d64c1044323f. 2023-02-11 03:55:05,566 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 614a4b4b2b88efbc57d9d64c1044323f, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676087689083.614a4b4b2b88efbc57d9d64c1044323f.', STARTKEY => '', ENDKEY => ''} 2023-02-11 03:55:05,567 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 614a4b4b2b88efbc57d9d64c1044323f 2023-02-11 03:55:05,567 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676087689083.614a4b4b2b88efbc57d9d64c1044323f.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-11 03:55:05,567 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 614a4b4b2b88efbc57d9d64c1044323f 2023-02-11 03:55:05,567 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 614a4b4b2b88efbc57d9d64c1044323f 2023-02-11 03:55:05,568 INFO [StoreOpener-614a4b4b2b88efbc57d9d64c1044323f-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 614a4b4b2b88efbc57d9d64c1044323f 2023-02-11 03:55:05,569 DEBUG [StoreOpener-614a4b4b2b88efbc57d9d64c1044323f-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/614a4b4b2b88efbc57d9d64c1044323f/family1 2023-02-11 03:55:05,569 DEBUG [StoreOpener-614a4b4b2b88efbc57d9d64c1044323f-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/614a4b4b2b88efbc57d9d64c1044323f/family1 2023-02-11 03:55:05,569 INFO [StoreOpener-614a4b4b2b88efbc57d9d64c1044323f-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 614a4b4b2b88efbc57d9d64c1044323f columnFamilyName family1 2023-02-11 03:55:05,574 DEBUG [StoreOpener-614a4b4b2b88efbc57d9d64c1044323f-1] regionserver.HStore(539): loaded hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/614a4b4b2b88efbc57d9d64c1044323f/family1/15da5851d7cc4e298e8f6ae30dec3114 2023-02-11 03:55:05,576 DEBUG [StoreOpener-614a4b4b2b88efbc57d9d64c1044323f-1] regionserver.HStore(539): loaded hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/614a4b4b2b88efbc57d9d64c1044323f/family1/ddc1531319754c3fb49047964998ab6c_SeqId_53_ 2023-02-11 03:55:05,577 INFO [StoreOpener-614a4b4b2b88efbc57d9d64c1044323f-1] regionserver.HStore(310): Store=614a4b4b2b88efbc57d9d64c1044323f/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 03:55:05,577 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/614a4b4b2b88efbc57d9d64c1044323f 2023-02-11 03:55:05,578 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/614a4b4b2b88efbc57d9d64c1044323f 2023-02-11 03:55:05,580 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 614a4b4b2b88efbc57d9d64c1044323f 2023-02-11 03:55:05,581 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 614a4b4b2b88efbc57d9d64c1044323f; next sequenceid=56; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=73669791, jitterRate=0.09776543080806732}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-11 03:55:05,581 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 614a4b4b2b88efbc57d9d64c1044323f: Running coprocessor pre-open hook at 1676087705567Writing region info on filesystem at 1676087705567Initializing all the Stores at 1676087705568 (+1 ms)Instantiating store for column family {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'DAILY'} at 1676087705568Cleaning up temporary data from old regions at 1676087705578 (+10 ms)Cleaning up detritus from prior splits at 1676087705579 (+1 ms)Running coprocessor post-open hooks at 1676087705581 (+2 ms)Region opened successfully at 1676087705581 2023-02-11 03:55:05,582 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676087689083.614a4b4b2b88efbc57d9d64c1044323f., pid=69, masterSystemTime=1676087705564 2023-02-11 03:55:05,583 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676087689083.614a4b4b2b88efbc57d9d64c1044323f. 2023-02-11 03:55:05,583 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676087689083.614a4b4b2b88efbc57d9d64c1044323f. 2023-02-11 03:55:05,584 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=67 updating hbase:meta row=614a4b4b2b88efbc57d9d64c1044323f, regionState=OPEN, openSeqNum=56, regionLocation=jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:55:05,584 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676087689083.614a4b4b2b88efbc57d9d64c1044323f.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1676087705583"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676087705583"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676087705583"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676087705583"}]},"ts":"1676087705583"} 2023-02-11 03:55:05,586 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=69, resume processing ppid=67 2023-02-11 03:55:05,586 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=69, ppid=67, state=SUCCESS; OpenRegionProcedure 614a4b4b2b88efbc57d9d64c1044323f, server=jenkins-hbase4.apache.org,37011,1676087618428 in 172 msec 2023-02-11 03:55:05,587 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=67, resume processing ppid=66 2023-02-11 03:55:05,587 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=67, ppid=66, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=614a4b4b2b88efbc57d9d64c1044323f, REOPEN/MOVE in 491 msec 2023-02-11 03:55:05,589 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=66, resume processing ppid=65 2023-02-11 03:55:05,589 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=66, ppid=65, state=SUCCESS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily in 495 msec 2023-02-11 03:55:05,591 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=65, state=SUCCESS; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily in 909 msec 2023-02-11 03:55:05,591 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] master.MasterRpcServices(1227): Checking to see if procedure is done pid=65 2023-02-11 03:55:05,592 INFO [Listener at localhost/46205] compactions.TestMobCompactor(278): alter status finished 2023-02-11 03:55:05,593 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=39921] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'DAILY'}] of the table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-11 03:55:05,594 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676087621935] procedure2.ProcedureExecutor(1029): Stored pid=70, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-02-11 03:55:05,595 DEBUG [PEWorker-4] locking.LockProcedure(309): LOCKED pid=70, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-02-11 03:55:05,595 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676087621935] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-11 03:55:05,595 INFO [PEWorker-4] procedure2.TimeoutExecutorThread(81): ADDED pid=70, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1676088305595 2023-02-11 03:55:05,596 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676087621935] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-02-11 03:55:05,596 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676087621935] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 4 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column 'family1' 2023-02-11 03:55:05,596 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676087621935] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-02-11 03:55:05,597 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676087621935] zookeeper.ReadOnlyZKClient(139): Connect 0x597a9812 to 127.0.0.1:53383 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-11 03:55:05,601 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676087621935] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@6c49275a, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-11 03:55:05,601 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151128 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-11 03:55:05,601 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151128 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-11 03:55:05,601 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151205 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-11 03:55:05,601 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151205 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-11 03:55:05,601 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20160103 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-11 03:55:05,601 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20160103 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-11 03:55:05,601 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-11 03:55:05,601 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-11 03:55:05,601 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676087621935] compactions.PartitionedMobCompactor(354): After compaction, there are 4 mob files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-02-11 03:55:05,602 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676087621935] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-11 03:55:05,602 DEBUG [PEWorker-3] locking.LockProcedure(242): UNLOCKED pid=70, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-02-11 03:55:05,603 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=70, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE in 8 msec 2023-02-11 03:55:05,653 INFO [Listener at localhost/46205] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily Thread=249 (was 241) - Thread LEAK? -, OpenFileDescriptor=640 (was 608) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=175 (was 159) - SystemLoadAverage LEAK? -, ProcessCount=168 (was 168), AvailableMemoryMB=3102 (was 3126) 2023-02-11 03:55:05,664 INFO [Listener at localhost/46205] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testGetAfterCompaction Thread=249, OpenFileDescriptor=640, MaxFileDescriptor=60000, SystemLoadAverage=175, ProcessCount=168, AvailableMemoryMB=3102 2023-02-11 03:55:05,666 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] master.HMaster$4(2102): Client=jenkins//172.31.14.131 create 'testGetAfterCompaction', {TABLE_ATTRIBUTES => {coprocessor$1 => '|org.apache.hadoop.hbase.mob.compactions.TestMobCompactor$CompactTwoLatestHfilesCopro|1073741823|'}}, {NAME => 'f1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} 2023-02-11 03:55:05,667 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] procedure2.ProcedureExecutor(1029): Stored pid=71, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testGetAfterCompaction 2023-02-11 03:55:05,668 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=71, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testGetAfterCompaction execute state=CREATE_TABLE_PRE_OPERATION 2023-02-11 03:55:05,668 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] master.MasterRpcServices(697): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testGetAfterCompaction" procId is: 71 2023-02-11 03:55:05,668 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=71, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testGetAfterCompaction execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-11 03:55:05,669 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] master.MasterRpcServices(1227): Checking to see if procedure is done pid=71 2023-02-11 03:55:05,670 DEBUG [HFileArchiver-11] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/.tmp/data/default/testGetAfterCompaction/dd87073924e61ec006a45079cb50294f 2023-02-11 03:55:05,670 DEBUG [HFileArchiver-11] backup.HFileArchiver(153): Directory hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/.tmp/data/default/testGetAfterCompaction/dd87073924e61ec006a45079cb50294f empty. 2023-02-11 03:55:05,670 DEBUG [HFileArchiver-11] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/.tmp/data/default/testGetAfterCompaction/dd87073924e61ec006a45079cb50294f 2023-02-11 03:55:05,670 DEBUG [PEWorker-5] procedure.DeleteTableProcedure(328): Archived testGetAfterCompaction regions 2023-02-11 03:55:05,770 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] master.MasterRpcServices(1227): Checking to see if procedure is done pid=71 2023-02-11 03:55:05,971 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] master.MasterRpcServices(1227): Checking to see if procedure is done pid=71 2023-02-11 03:55:06,077 DEBUG [PEWorker-5] util.FSTableDescriptors(570): Wrote into hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/.tmp/data/default/testGetAfterCompaction/.tabledesc/.tableinfo.0000000001 2023-02-11 03:55:06,079 INFO [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(7671): creating {ENCODED => dd87073924e61ec006a45079cb50294f, NAME => 'testGetAfterCompaction,,1676087705666.dd87073924e61ec006a45079cb50294f.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testGetAfterCompaction', {TABLE_ATTRIBUTES => {coprocessor$1 => '|org.apache.hadoop.hbase.mob.compactions.TestMobCompactor$CompactTwoLatestHfilesCopro|1073741823|'}}, {NAME => 'f1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/.tmp 2023-02-11 03:55:06,271 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] master.MasterRpcServices(1227): Checking to see if procedure is done pid=71 2023-02-11 03:55:06,484 DEBUG [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(865): Instantiated testGetAfterCompaction,,1676087705666.dd87073924e61ec006a45079cb50294f.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-11 03:55:06,485 DEBUG [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(1603): Closing dd87073924e61ec006a45079cb50294f, disabling compactions & flushes 2023-02-11 03:55:06,485 INFO [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(1625): Closing region testGetAfterCompaction,,1676087705666.dd87073924e61ec006a45079cb50294f. 2023-02-11 03:55:06,485 DEBUG [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testGetAfterCompaction,,1676087705666.dd87073924e61ec006a45079cb50294f. 2023-02-11 03:55:06,485 DEBUG [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(1713): Acquired close lock on testGetAfterCompaction,,1676087705666.dd87073924e61ec006a45079cb50294f. after waiting 0 ms 2023-02-11 03:55:06,485 DEBUG [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(1723): Updates disabled for region testGetAfterCompaction,,1676087705666.dd87073924e61ec006a45079cb50294f. 2023-02-11 03:55:06,485 INFO [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(1837): Closed testGetAfterCompaction,,1676087705666.dd87073924e61ec006a45079cb50294f. 2023-02-11 03:55:06,485 DEBUG [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(1557): Region close journal for dd87073924e61ec006a45079cb50294f: Waiting for close lock at 1676087706484Disabling compacts and flushes for region at 1676087706484Disabling writes for close at 1676087706485 (+1 ms)Writing region close event to WAL at 1676087706485Closed at 1676087706485 2023-02-11 03:55:06,487 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=71, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testGetAfterCompaction execute state=CREATE_TABLE_ADD_TO_META 2023-02-11 03:55:06,488 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testGetAfterCompaction,,1676087705666.dd87073924e61ec006a45079cb50294f.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676087706488"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676087706488"}]},"ts":"1676087706488"} 2023-02-11 03:55:06,489 INFO [PEWorker-5] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-11 03:55:06,489 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=71, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testGetAfterCompaction execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-11 03:55:06,490 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testGetAfterCompaction","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676087706490"}]},"ts":"1676087706490"} 2023-02-11 03:55:06,490 INFO [PEWorker-5] hbase.MetaTableAccessor(1635): Updated tableName=testGetAfterCompaction, state=ENABLING in hbase:meta 2023-02-11 03:55:06,493 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=72, ppid=71, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testGetAfterCompaction, region=dd87073924e61ec006a45079cb50294f, ASSIGN}] 2023-02-11 03:55:06,494 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=72, ppid=71, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testGetAfterCompaction, region=dd87073924e61ec006a45079cb50294f, ASSIGN 2023-02-11 03:55:06,495 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=72, ppid=71, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testGetAfterCompaction, region=dd87073924e61ec006a45079cb50294f, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,37011,1676087618428; forceNewPlan=false, retain=false 2023-02-11 03:55:06,646 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=72 updating hbase:meta row=dd87073924e61ec006a45079cb50294f, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:55:06,646 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testGetAfterCompaction,,1676087705666.dd87073924e61ec006a45079cb50294f.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676087706646"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676087706646"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676087706646"}]},"ts":"1676087706646"} 2023-02-11 03:55:06,647 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=73, ppid=72, state=RUNNABLE; OpenRegionProcedure dd87073924e61ec006a45079cb50294f, server=jenkins-hbase4.apache.org,37011,1676087618428}] 2023-02-11 03:55:06,772 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] master.MasterRpcServices(1227): Checking to see if procedure is done pid=71 2023-02-11 03:55:06,801 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testGetAfterCompaction,,1676087705666.dd87073924e61ec006a45079cb50294f. 2023-02-11 03:55:06,801 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => dd87073924e61ec006a45079cb50294f, NAME => 'testGetAfterCompaction,,1676087705666.dd87073924e61ec006a45079cb50294f.', STARTKEY => '', ENDKEY => ''} 2023-02-11 03:55:06,801 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(215): Loading coprocessor class org.apache.hadoop.hbase.mob.compactions.TestMobCompactor$CompactTwoLatestHfilesCopro with path null and priority 1073741823 2023-02-11 03:55:06,803 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.RegionCoprocessorHost(393): Loaded coprocessor org.apache.hadoop.hbase.mob.compactions.TestMobCompactor$CompactTwoLatestHfilesCopro from HTD of testGetAfterCompaction successfully. 2023-02-11 03:55:06,803 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testGetAfterCompaction dd87073924e61ec006a45079cb50294f 2023-02-11 03:55:06,803 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testGetAfterCompaction,,1676087705666.dd87073924e61ec006a45079cb50294f.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-11 03:55:06,803 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for dd87073924e61ec006a45079cb50294f 2023-02-11 03:55:06,803 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for dd87073924e61ec006a45079cb50294f 2023-02-11 03:55:06,804 INFO [StoreOpener-dd87073924e61ec006a45079cb50294f-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family f1 of region dd87073924e61ec006a45079cb50294f 2023-02-11 03:55:06,806 DEBUG [StoreOpener-dd87073924e61ec006a45079cb50294f-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testGetAfterCompaction/dd87073924e61ec006a45079cb50294f/f1 2023-02-11 03:55:06,806 DEBUG [StoreOpener-dd87073924e61ec006a45079cb50294f-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testGetAfterCompaction/dd87073924e61ec006a45079cb50294f/f1 2023-02-11 03:55:06,806 INFO [StoreOpener-dd87073924e61ec006a45079cb50294f-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region dd87073924e61ec006a45079cb50294f columnFamilyName f1 2023-02-11 03:55:06,807 INFO [StoreOpener-dd87073924e61ec006a45079cb50294f-1] regionserver.HStore(310): Store=dd87073924e61ec006a45079cb50294f/f1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 03:55:06,808 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testGetAfterCompaction/dd87073924e61ec006a45079cb50294f 2023-02-11 03:55:06,808 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testGetAfterCompaction/dd87073924e61ec006a45079cb50294f 2023-02-11 03:55:06,810 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for dd87073924e61ec006a45079cb50294f 2023-02-11 03:55:06,812 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testGetAfterCompaction/dd87073924e61ec006a45079cb50294f/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-11 03:55:06,812 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened dd87073924e61ec006a45079cb50294f; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=70893917, jitterRate=0.05640168488025665}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-11 03:55:06,812 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for dd87073924e61ec006a45079cb50294f: Running coprocessor pre-open hook at 1676087706803Writing region info on filesystem at 1676087706803Initializing all the Stores at 1676087706804 (+1 ms)Instantiating store for column family {NAME => 'f1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1676087706804Cleaning up temporary data from old regions at 1676087706808 (+4 ms)Cleaning up detritus from prior splits at 1676087706809 (+1 ms)Running coprocessor post-open hooks at 1676087706812 (+3 ms)Region opened successfully at 1676087706812 2023-02-11 03:55:06,813 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testGetAfterCompaction,,1676087705666.dd87073924e61ec006a45079cb50294f., pid=73, masterSystemTime=1676087706799 2023-02-11 03:55:06,814 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testGetAfterCompaction,,1676087705666.dd87073924e61ec006a45079cb50294f. 2023-02-11 03:55:06,814 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testGetAfterCompaction,,1676087705666.dd87073924e61ec006a45079cb50294f. 2023-02-11 03:55:06,814 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=72 updating hbase:meta row=dd87073924e61ec006a45079cb50294f, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:55:06,815 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testGetAfterCompaction,,1676087705666.dd87073924e61ec006a45079cb50294f.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676087706814"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676087706814"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676087706814"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676087706814"}]},"ts":"1676087706814"} 2023-02-11 03:55:06,817 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=73, resume processing ppid=72 2023-02-11 03:55:06,818 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=73, ppid=72, state=SUCCESS; OpenRegionProcedure dd87073924e61ec006a45079cb50294f, server=jenkins-hbase4.apache.org,37011,1676087618428 in 169 msec 2023-02-11 03:55:06,819 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=72, resume processing ppid=71 2023-02-11 03:55:06,819 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=72, ppid=71, state=SUCCESS; TransitRegionStateProcedure table=testGetAfterCompaction, region=dd87073924e61ec006a45079cb50294f, ASSIGN in 325 msec 2023-02-11 03:55:06,820 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=71, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testGetAfterCompaction execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-11 03:55:06,820 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testGetAfterCompaction","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676087706820"}]},"ts":"1676087706820"} 2023-02-11 03:55:06,821 INFO [PEWorker-2] hbase.MetaTableAccessor(1635): Updated tableName=testGetAfterCompaction, state=ENABLED in hbase:meta 2023-02-11 03:55:06,824 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=71, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testGetAfterCompaction execute state=CREATE_TABLE_POST_OPERATION 2023-02-11 03:55:06,825 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=71, state=SUCCESS; CreateTableProcedure table=testGetAfterCompaction in 1.1580 sec 2023-02-11 03:55:07,475 DEBUG [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(130): Registering adapter for the MetricRegistry: RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.mob.compactions.TestMobCompactor$CompactTwoLatestHfilesCopro 2023-02-11 03:55:07,475 INFO [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(134): Registering RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.mob.compactions.TestMobCompactor$CompactTwoLatestHfilesCopro Metrics about HBase RegionObservers 2023-02-11 03:55:07,773 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] master.MasterRpcServices(1227): Checking to see if procedure is done pid=71 2023-02-11 03:55:07,773 INFO [Listener at localhost/46205] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testGetAfterCompaction, procId: 71 completed 2023-02-11 03:55:07,774 DEBUG [Listener at localhost/46205] hbase.HBaseTestingUtility(3430): Waiting until all regions of table testGetAfterCompaction get assigned. Timeout = 60000ms 2023-02-11 03:55:07,775 INFO [Listener at localhost/46205] hbase.Waiter(180): Waiting up to [60,000] milli-secs(wait.for.ratio=[1]) 2023-02-11 03:55:07,778 INFO [Listener at localhost/46205] hbase.HBaseTestingUtility(3484): All regions for table testGetAfterCompaction assigned to meta. Checking AM states. 2023-02-11 03:55:07,779 INFO [Listener at localhost/46205] hbase.Waiter(180): Waiting up to [60,000] milli-secs(wait.for.ratio=[1]) 2023-02-11 03:55:07,779 INFO [Listener at localhost/46205] hbase.HBaseTestingUtility(3504): All regions for table testGetAfterCompaction assigned. 2023-02-11 03:55:07,782 INFO [Listener at localhost/46205] regionserver.HRegion(2744): Flushing dd87073924e61ec006a45079cb50294f 1/1 column families, dataSize=37 B heapSize=368 B 2023-02-11 03:55:08,194 INFO [Listener at localhost/46205] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230211d467fc3723694fb290dc60e9c482afff to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e20230211d467fc3723694fb290dc60e9c482afff 2023-02-11 03:55:08,454 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-02-11 03:55:08,454 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x38bb69c6 to 127.0.0.1:53383 2023-02-11 03:55:08,454 DEBUG [Finalizer] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-11 03:55:08,455 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x597a9812 to 127.0.0.1:53383 2023-02-11 03:55:08,455 DEBUG [Finalizer] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-11 03:55:08,455 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-02-11 03:55:08,455 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x6be34b78 to 127.0.0.1:53383 2023-02-11 03:55:08,455 DEBUG [Finalizer] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-11 03:55:08,455 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-02-11 03:55:08,455 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x48890c4d to 127.0.0.1:53383 2023-02-11 03:55:08,455 DEBUG [Finalizer] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-11 03:55:08,455 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x644ac847 to 127.0.0.1:53383 2023-02-11 03:55:08,455 DEBUG [Finalizer] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-11 03:55:08,456 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-02-11 03:55:08,456 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x79fa0667 to 127.0.0.1:53383 2023-02-11 03:55:08,456 DEBUG [Finalizer] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-11 03:55:08,599 INFO [Listener at localhost/46205] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=37, hasBloomFilter=true, into tmp file hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testGetAfterCompaction/dd87073924e61ec006a45079cb50294f/.tmp/f1/eb2ad76a5f2b4a9ca525eb52075ce7e2 2023-02-11 03:55:08,604 DEBUG [Listener at localhost/46205] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testGetAfterCompaction/dd87073924e61ec006a45079cb50294f/.tmp/f1/eb2ad76a5f2b4a9ca525eb52075ce7e2 as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testGetAfterCompaction/dd87073924e61ec006a45079cb50294f/f1/eb2ad76a5f2b4a9ca525eb52075ce7e2 2023-02-11 03:55:08,607 INFO [Listener at localhost/46205] regionserver.HStore(1080): Added hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testGetAfterCompaction/dd87073924e61ec006a45079cb50294f/f1/eb2ad76a5f2b4a9ca525eb52075ce7e2, entries=1, sequenceid=5, filesize=4.9 K 2023-02-11 03:55:08,608 INFO [Listener at localhost/46205] regionserver.HRegion(2947): Finished flush of dataSize ~37 B/37, heapSize ~352 B/352, currentSize=0 B/0 for dd87073924e61ec006a45079cb50294f in 826ms, sequenceid=5, compaction requested=false 2023-02-11 03:55:08,608 DEBUG [Listener at localhost/46205] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testGetAfterCompaction' 2023-02-11 03:55:08,609 DEBUG [Listener at localhost/46205] regionserver.HRegion(2445): Flush status journal for dd87073924e61ec006a45079cb50294f: 2023-02-11 03:55:08,610 INFO [Listener at localhost/46205] regionserver.HRegion(2744): Flushing dd87073924e61ec006a45079cb50294f 1/1 column families, dataSize=37 B heapSize=368 B 2023-02-11 03:55:08,967 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-11 03:55:09,021 INFO [Listener at localhost/46205] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230211272bbe882b8140a2a3effaca90c27924 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e20230211272bbe882b8140a2a3effaca90c27924 2023-02-11 03:55:09,427 INFO [Listener at localhost/46205] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=37, hasBloomFilter=true, into tmp file hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testGetAfterCompaction/dd87073924e61ec006a45079cb50294f/.tmp/f1/726e888f51fa43119551a427b0a3e8be 2023-02-11 03:55:09,432 DEBUG [Listener at localhost/46205] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testGetAfterCompaction/dd87073924e61ec006a45079cb50294f/.tmp/f1/726e888f51fa43119551a427b0a3e8be as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testGetAfterCompaction/dd87073924e61ec006a45079cb50294f/f1/726e888f51fa43119551a427b0a3e8be 2023-02-11 03:55:09,435 INFO [Listener at localhost/46205] regionserver.HStore(1080): Added hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testGetAfterCompaction/dd87073924e61ec006a45079cb50294f/f1/726e888f51fa43119551a427b0a3e8be, entries=1, sequenceid=9, filesize=4.9 K 2023-02-11 03:55:09,436 INFO [Listener at localhost/46205] regionserver.HRegion(2947): Finished flush of dataSize ~37 B/37, heapSize ~352 B/352, currentSize=0 B/0 for dd87073924e61ec006a45079cb50294f in 826ms, sequenceid=9, compaction requested=false 2023-02-11 03:55:09,436 DEBUG [Listener at localhost/46205] regionserver.HRegion(2445): Flush status journal for dd87073924e61ec006a45079cb50294f: 2023-02-11 03:55:09,437 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=39921] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'f1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}] of the table testGetAfterCompaction 2023-02-11 03:55:09,439 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676087621935] procedure2.ProcedureExecutor(1029): Stored pid=74, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testGetAfterCompaction.mobLock, type=EXCLUSIVE 2023-02-11 03:55:09,439 DEBUG [PEWorker-1] locking.LockProcedure(309): LOCKED pid=74, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testGetAfterCompaction.mobLock, type=EXCLUSIVE 2023-02-11 03:55:09,440 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676087621935] mob.MobUtils(833): start MOB compaction of files for table='testGetAfterCompaction', column='f1', allFiles=false, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-11 03:55:09,440 INFO [PEWorker-1] procedure2.TimeoutExecutorThread(81): ADDED pid=74, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testGetAfterCompaction.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1676088309440 2023-02-11 03:55:09,440 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676087621935] compactions.PartitionedMobCompactor(142): is allFiles: false 2023-02-11 03:55:09,440 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676087621935] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 2 selected files, and 0 irrelevant files table 'testGetAfterCompaction' and column 'f1' 2023-02-11 03:55:09,440 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676087621935] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testGetAfterCompaction' column='f1' 2023-02-11 03:55:09,441 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676087621935] zookeeper.ReadOnlyZKClient(139): Connect 0x369d21bb to 127.0.0.1:53383 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-11 03:55:09,446 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676087621935] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@13cba3c5, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-11 03:55:09,446 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20230211 for table='testGetAfterCompaction' and column='f1' 2023-02-11 03:55:09,489 INFO [Listener at localhost/46205] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testGetAfterCompaction/dd87073924e61ec006a45079cb50294f/f1/726e888f51fa43119551a427b0a3e8be; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676087709427; access_time=1676087708612; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-11 03:55:09,489 INFO [Listener at localhost/46205] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testGetAfterCompaction/dd87073924e61ec006a45079cb50294f/f1/eb2ad76a5f2b4a9ca525eb52075ce7e2; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676087708599; access_time=1676087707785; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-11 03:55:09,539 INFO [Listener at localhost/46205] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testGetAfterCompaction/dd87073924e61ec006a45079cb50294f/f1/726e888f51fa43119551a427b0a3e8be; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676087709427; access_time=1676087708612; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-11 03:55:09,540 INFO [Listener at localhost/46205] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testGetAfterCompaction/dd87073924e61ec006a45079cb50294f/f1/eb2ad76a5f2b4a9ca525eb52075ce7e2; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676087708599; access_time=1676087707785; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-11 03:55:09,590 INFO [Listener at localhost/46205] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testGetAfterCompaction/dd87073924e61ec006a45079cb50294f/f1/726e888f51fa43119551a427b0a3e8be; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676087709427; access_time=1676087708612; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-11 03:55:09,590 INFO [Listener at localhost/46205] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testGetAfterCompaction/dd87073924e61ec006a45079cb50294f/f1/eb2ad76a5f2b4a9ca525eb52075ce7e2; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676087708599; access_time=1676087707785; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-11 03:55:09,641 INFO [Listener at localhost/46205] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testGetAfterCompaction/dd87073924e61ec006a45079cb50294f/f1/726e888f51fa43119551a427b0a3e8be; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676087709427; access_time=1676087708612; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-11 03:55:09,641 INFO [Listener at localhost/46205] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testGetAfterCompaction/dd87073924e61ec006a45079cb50294f/f1/eb2ad76a5f2b4a9ca525eb52075ce7e2; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676087708599; access_time=1676087707785; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-11 03:55:09,692 INFO [Listener at localhost/46205] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testGetAfterCompaction/dd87073924e61ec006a45079cb50294f/f1/726e888f51fa43119551a427b0a3e8be; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676087709427; access_time=1676087708612; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-11 03:55:09,692 INFO [Listener at localhost/46205] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testGetAfterCompaction/dd87073924e61ec006a45079cb50294f/f1/eb2ad76a5f2b4a9ca525eb52075ce7e2; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676087708599; access_time=1676087707785; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-11 03:55:09,742 INFO [Listener at localhost/46205] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testGetAfterCompaction/dd87073924e61ec006a45079cb50294f/f1/726e888f51fa43119551a427b0a3e8be; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676087709427; access_time=1676087708612; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-11 03:55:09,742 INFO [Listener at localhost/46205] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testGetAfterCompaction/dd87073924e61ec006a45079cb50294f/f1/eb2ad76a5f2b4a9ca525eb52075ce7e2; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676087708599; access_time=1676087707785; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-11 03:55:09,793 INFO [Listener at localhost/46205] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testGetAfterCompaction/dd87073924e61ec006a45079cb50294f/f1/726e888f51fa43119551a427b0a3e8be; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676087709427; access_time=1676087708612; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-11 03:55:09,793 INFO [Listener at localhost/46205] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testGetAfterCompaction/dd87073924e61ec006a45079cb50294f/f1/eb2ad76a5f2b4a9ca525eb52075ce7e2; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676087708599; access_time=1676087707785; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-11 03:55:09,844 INFO [Listener at localhost/46205] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testGetAfterCompaction/dd87073924e61ec006a45079cb50294f/f1/726e888f51fa43119551a427b0a3e8be; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676087709427; access_time=1676087708612; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-11 03:55:09,844 INFO [Listener at localhost/46205] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testGetAfterCompaction/dd87073924e61ec006a45079cb50294f/f1/eb2ad76a5f2b4a9ca525eb52075ce7e2; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676087708599; access_time=1676087707785; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-11 03:55:09,895 INFO [Listener at localhost/46205] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testGetAfterCompaction/dd87073924e61ec006a45079cb50294f/f1/726e888f51fa43119551a427b0a3e8be; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676087709427; access_time=1676087708612; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-11 03:55:09,895 INFO [Listener at localhost/46205] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testGetAfterCompaction/dd87073924e61ec006a45079cb50294f/f1/eb2ad76a5f2b4a9ca525eb52075ce7e2; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676087708599; access_time=1676087707785; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-11 03:55:09,945 INFO [Listener at localhost/46205] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testGetAfterCompaction/dd87073924e61ec006a45079cb50294f/f1/726e888f51fa43119551a427b0a3e8be; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676087709427; access_time=1676087708612; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-11 03:55:09,945 INFO [Listener at localhost/46205] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testGetAfterCompaction/dd87073924e61ec006a45079cb50294f/f1/eb2ad76a5f2b4a9ca525eb52075ce7e2; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676087708599; access_time=1676087707785; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-11 03:55:09,996 INFO [Listener at localhost/46205] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testGetAfterCompaction/dd87073924e61ec006a45079cb50294f/f1/726e888f51fa43119551a427b0a3e8be; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676087709427; access_time=1676087708612; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-11 03:55:09,996 INFO [Listener at localhost/46205] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testGetAfterCompaction/dd87073924e61ec006a45079cb50294f/f1/eb2ad76a5f2b4a9ca525eb52075ce7e2; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676087708599; access_time=1676087707785; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-11 03:55:10,047 INFO [Listener at localhost/46205] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testGetAfterCompaction/dd87073924e61ec006a45079cb50294f/f1/726e888f51fa43119551a427b0a3e8be; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676087709427; access_time=1676087708612; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-11 03:55:10,047 INFO [Listener at localhost/46205] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testGetAfterCompaction/dd87073924e61ec006a45079cb50294f/f1/eb2ad76a5f2b4a9ca525eb52075ce7e2; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676087708599; access_time=1676087707785; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-11 03:55:10,098 INFO [Listener at localhost/46205] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testGetAfterCompaction/dd87073924e61ec006a45079cb50294f/f1/726e888f51fa43119551a427b0a3e8be; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676087709427; access_time=1676087708612; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-11 03:55:10,098 INFO [Listener at localhost/46205] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testGetAfterCompaction/dd87073924e61ec006a45079cb50294f/f1/eb2ad76a5f2b4a9ca525eb52075ce7e2; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676087708599; access_time=1676087707785; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-11 03:55:10,149 INFO [Listener at localhost/46205] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testGetAfterCompaction/dd87073924e61ec006a45079cb50294f/f1/726e888f51fa43119551a427b0a3e8be; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676087709427; access_time=1676087708612; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-11 03:55:10,149 INFO [Listener at localhost/46205] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testGetAfterCompaction/dd87073924e61ec006a45079cb50294f/f1/eb2ad76a5f2b4a9ca525eb52075ce7e2; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676087708599; access_time=1676087707785; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-11 03:55:10,199 INFO [Listener at localhost/46205] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testGetAfterCompaction/dd87073924e61ec006a45079cb50294f/f1/726e888f51fa43119551a427b0a3e8be; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676087709427; access_time=1676087708612; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-11 03:55:10,200 INFO [Listener at localhost/46205] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testGetAfterCompaction/dd87073924e61ec006a45079cb50294f/f1/eb2ad76a5f2b4a9ca525eb52075ce7e2; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676087708599; access_time=1676087707785; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-11 03:55:10,250 INFO [Listener at localhost/46205] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testGetAfterCompaction/dd87073924e61ec006a45079cb50294f/f1/726e888f51fa43119551a427b0a3e8be; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676087709427; access_time=1676087708612; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-11 03:55:10,250 INFO [Listener at localhost/46205] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testGetAfterCompaction/dd87073924e61ec006a45079cb50294f/f1/eb2ad76a5f2b4a9ca525eb52075ce7e2; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676087708599; access_time=1676087707785; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-11 03:55:10,268 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202302114242fde04ccf43e99cbffd21094abe57 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e202302114242fde04ccf43e99cbffd21094abe57 2023-02-11 03:55:10,268 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20230211 table='testGetAfterCompaction' column='f1' 2023-02-11 03:55:10,281 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-11 03:55:10,282 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:36746, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-11 03:55:10,288 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-02-11 03:55:10,289 INFO [RS-EventLoopGroup-1-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:43540, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-02-11 03:55:10,298 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/.bulkload/default/testGetAfterCompaction/d41d8cd98f00b204e9800998ecf8427e20230211/f1/9c3dd96f9a7d4c5aa22f8c1383715082 first=Optional[r1] last=Optional[r2] 2023-02-11 03:55:10,298 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testGetAfterCompaction,,1676087705666.dd87073924e61ec006a45079cb50294f., hostname=jenkins-hbase4.apache.org,37011,1676087618428, seqNum=2 for row with hfile group [{f1,hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/.bulkload/default/testGetAfterCompaction/d41d8cd98f00b204e9800998ecf8427e20230211/f1/9c3dd96f9a7d4c5aa22f8c1383715082}] 2023-02-11 03:55:10,301 INFO [Listener at localhost/46205] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testGetAfterCompaction/dd87073924e61ec006a45079cb50294f/f1/726e888f51fa43119551a427b0a3e8be; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676087709427; access_time=1676087708612; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-11 03:55:10,301 INFO [Listener at localhost/46205] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testGetAfterCompaction/dd87073924e61ec006a45079cb50294f/f1/eb2ad76a5f2b4a9ca525eb52075ce7e2; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676087708599; access_time=1676087707785; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-11 03:55:10,302 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.HStore(763): Validating hfile at hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/.bulkload/default/testGetAfterCompaction/d41d8cd98f00b204e9800998ecf8427e20230211/f1/9c3dd96f9a7d4c5aa22f8c1383715082 for inclusion in dd87073924e61ec006a45079cb50294f/f1 2023-02-11 03:55:10,303 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.HStore(775): HFile bounds: first=r1 last=r2 2023-02-11 03:55:10,304 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.HStore(777): Region bounds: first= last= 2023-02-11 03:55:10,304 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.HRegion(2445): Flush status journal for dd87073924e61ec006a45079cb50294f: 2023-02-11 03:55:10,304 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/.bulkload/default/testGetAfterCompaction/d41d8cd98f00b204e9800998ecf8427e20230211/f1/9c3dd96f9a7d4c5aa22f8c1383715082 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/staging/jenkins__testGetAfterCompaction__gks1sgof906vlqtmj1ldupasv7iql3odd046ubvsomrtlaaoo1v8t4t9oeva6hfj/f1/9c3dd96f9a7d4c5aa22f8c1383715082 2023-02-11 03:55:10,306 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/staging/jenkins__testGetAfterCompaction__gks1sgof906vlqtmj1ldupasv7iql3odd046ubvsomrtlaaoo1v8t4t9oeva6hfj/f1/9c3dd96f9a7d4c5aa22f8c1383715082 as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testGetAfterCompaction/dd87073924e61ec006a45079cb50294f/f1/a24e770467ae446b921aa8dfce808122_SeqId_12_ 2023-02-11 03:55:10,307 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.HStore(849): Loaded HFile hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/staging/jenkins__testGetAfterCompaction__gks1sgof906vlqtmj1ldupasv7iql3odd046ubvsomrtlaaoo1v8t4t9oeva6hfj/f1/9c3dd96f9a7d4c5aa22f8c1383715082 into dd87073924e61ec006a45079cb50294f/f1 as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testGetAfterCompaction/dd87073924e61ec006a45079cb50294f/f1/a24e770467ae446b921aa8dfce808122_SeqId_12_ - updating store file list. 2023-02-11 03:55:10,309 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.HStore(882): Loaded HFile hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testGetAfterCompaction/dd87073924e61ec006a45079cb50294f/f1/a24e770467ae446b921aa8dfce808122_SeqId_12_ into dd87073924e61ec006a45079cb50294f/f1 2023-02-11 03:55:10,309 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.HStore(855): Successfully loaded hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/staging/jenkins__testGetAfterCompaction__gks1sgof906vlqtmj1ldupasv7iql3odd046ubvsomrtlaaoo1v8t4t9oeva6hfj/f1/9c3dd96f9a7d4c5aa22f8c1383715082 into dd87073924e61ec006a45079cb50294f/f1 (new location: hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testGetAfterCompaction/dd87073924e61ec006a45079cb50294f/f1/a24e770467ae446b921aa8dfce808122_SeqId_12_) 2023-02-11 03:55:10,310 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/staging/jenkins__testGetAfterCompaction__gks1sgof906vlqtmj1ldupasv7iql3odd046ubvsomrtlaaoo1v8t4t9oeva6hfj/f1/9c3dd96f9a7d4c5aa22f8c1383715082 2023-02-11 03:55:10,310 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.HStore(1912): dd87073924e61ec006a45079cb50294f/f1 is initiating minor compaction 2023-02-11 03:55:10,310 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@11d0f135; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-11 03:55:10,310 INFO [RS:0;jenkins-hbase4:37011-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of dd87073924e61ec006a45079cb50294f/f1 in testGetAfterCompaction,,1676087705666.dd87073924e61ec006a45079cb50294f. 2023-02-11 03:55:10,310 INFO [RS:0;jenkins-hbase4:37011-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testGetAfterCompaction/dd87073924e61ec006a45079cb50294f/f1/726e888f51fa43119551a427b0a3e8be, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testGetAfterCompaction/dd87073924e61ec006a45079cb50294f/f1/a24e770467ae446b921aa8dfce808122_SeqId_12_] into tmpdir=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testGetAfterCompaction/dd87073924e61ec006a45079cb50294f/.tmp, totalSize=10.0 K 2023-02-11 03:55:10,310 DEBUG [RS:0;jenkins-hbase4:37011-shortCompactions-0] compactions.Compactor(207): Compacting 726e888f51fa43119551a427b0a3e8be, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=9 2023-02-11 03:55:10,310 DEBUG [RS:0;jenkins-hbase4:37011-shortCompactions-0] compactions.Compactor(207): Compacting a24e770467ae446b921aa8dfce808122_SeqId_12_, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13 2023-02-11 03:55:10,312 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20230211 table='testGetAfterCompaction' column='f1' 2023-02-11 03:55:10,313 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-11 03:55:10,319 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e20230211272bbe882b8140a2a3effaca90c27924 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e20230211272bbe882b8140a2a3effaca90c27924 2023-02-11 03:55:10,320 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e20230211d467fc3723694fb290dc60e9c482afff to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e20230211d467fc3723694fb290dc60e9c482afff 2023-02-11 03:55:10,320 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testGetAfterCompaction 2 expired mob files are deleted 2023-02-11 03:55:10,320 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/.bulkload/default/testGetAfterCompaction/d41d8cd98f00b204e9800998ecf8427e20230211' 2023-02-11 03:55:10,321 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 2 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20230211 for table='testGetAfterCompaction' and column='f1' 2023-02-11 03:55:10,321 INFO [RS:0;jenkins-hbase4:37011-shortCompactions-0] throttle.PressureAwareThroughputController(145): dd87073924e61ec006a45079cb50294f#f1#compaction#76 average throughput is 0.13 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 61.54 MB/second 2023-02-11 03:55:10,321 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676087621935] compactions.PartitionedMobCompactor(354): After compaction, there are 1 mob files. table='testGetAfterCompaction' column='f1' 2023-02-11 03:55:10,321 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676087621935] mob.MobUtils(840): end MOB compaction of files for table='testGetAfterCompaction', column='f1', allFiles=false, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-11 03:55:10,322 DEBUG [PEWorker-4] locking.LockProcedure(242): UNLOCKED pid=74, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testGetAfterCompaction.mobLock, type=EXCLUSIVE 2023-02-11 03:55:10,323 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=74, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testGetAfterCompaction.mobLock, type=EXCLUSIVE in 884 msec 2023-02-11 03:55:10,352 INFO [Listener at localhost/46205] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testGetAfterCompaction/dd87073924e61ec006a45079cb50294f/f1/726e888f51fa43119551a427b0a3e8be; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676087709427; access_time=1676087708612; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-11 03:55:10,352 INFO [Listener at localhost/46205] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testGetAfterCompaction/dd87073924e61ec006a45079cb50294f/f1/a24e770467ae446b921aa8dfce808122_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1676087710264; access_time=1676087709456; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-02-11 03:55:10,352 INFO [Listener at localhost/46205] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testGetAfterCompaction/dd87073924e61ec006a45079cb50294f/f1/eb2ad76a5f2b4a9ca525eb52075ce7e2; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676087708599; access_time=1676087707785; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-11 03:55:10,360 DEBUG [Listener at localhost/46205] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-02-11 03:55:10,364 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:36752, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-02-11 03:55:10,364 INFO [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=37011] regionserver.RSRpcServices(1662): Compacting testGetAfterCompaction,,1676087705666.dd87073924e61ec006a45079cb50294f. 2023-02-11 03:55:10,365 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=37011] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 2 compacting, 0 eligible, 16 blocking 2023-02-11 03:55:10,365 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=37011] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 0 files of size 0 starting at candidate #-1 after considering 0 permutations with 0 in ratio 2023-02-11 03:55:10,365 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=37011] compactions.SortedCompactionPolicy(232): Not compacting files because we only have 0 files ready for compaction. Need 3 to initiate. 2023-02-11 03:55:10,365 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=37011] regionserver.CompactSplit(415): Not compacting testGetAfterCompaction,,1676087705666.dd87073924e61ec006a45079cb50294f. because compaction request was cancelled 2023-02-11 03:55:10,416 INFO [Listener at localhost/46205] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testGetAfterCompaction/dd87073924e61ec006a45079cb50294f/f1/726e888f51fa43119551a427b0a3e8be; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676087709427; access_time=1676087708612; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-11 03:55:10,417 INFO [Listener at localhost/46205] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testGetAfterCompaction/dd87073924e61ec006a45079cb50294f/f1/a24e770467ae446b921aa8dfce808122_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1676087710264; access_time=1676087709456; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-02-11 03:55:10,417 INFO [Listener at localhost/46205] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testGetAfterCompaction/dd87073924e61ec006a45079cb50294f/f1/eb2ad76a5f2b4a9ca525eb52075ce7e2; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676087708599; access_time=1676087707785; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-11 03:55:10,467 INFO [Listener at localhost/46205] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testGetAfterCompaction/dd87073924e61ec006a45079cb50294f/f1/726e888f51fa43119551a427b0a3e8be; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676087709427; access_time=1676087708612; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-11 03:55:10,468 INFO [Listener at localhost/46205] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testGetAfterCompaction/dd87073924e61ec006a45079cb50294f/f1/a24e770467ae446b921aa8dfce808122_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1676087710264; access_time=1676087709456; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-02-11 03:55:10,468 INFO [Listener at localhost/46205] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testGetAfterCompaction/dd87073924e61ec006a45079cb50294f/f1/eb2ad76a5f2b4a9ca525eb52075ce7e2; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676087708599; access_time=1676087707785; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-11 03:55:10,518 INFO [Listener at localhost/46205] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testGetAfterCompaction/dd87073924e61ec006a45079cb50294f/f1/726e888f51fa43119551a427b0a3e8be; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676087709427; access_time=1676087708612; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-11 03:55:10,518 INFO [Listener at localhost/46205] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testGetAfterCompaction/dd87073924e61ec006a45079cb50294f/f1/a24e770467ae446b921aa8dfce808122_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1676087710264; access_time=1676087709456; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-02-11 03:55:10,518 INFO [Listener at localhost/46205] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testGetAfterCompaction/dd87073924e61ec006a45079cb50294f/f1/eb2ad76a5f2b4a9ca525eb52075ce7e2; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676087708599; access_time=1676087707785; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-11 03:55:10,569 INFO [Listener at localhost/46205] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testGetAfterCompaction/dd87073924e61ec006a45079cb50294f/f1/726e888f51fa43119551a427b0a3e8be; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676087709427; access_time=1676087708612; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-11 03:55:10,569 INFO [Listener at localhost/46205] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testGetAfterCompaction/dd87073924e61ec006a45079cb50294f/f1/a24e770467ae446b921aa8dfce808122_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1676087710264; access_time=1676087709456; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-02-11 03:55:10,569 INFO [Listener at localhost/46205] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testGetAfterCompaction/dd87073924e61ec006a45079cb50294f/f1/eb2ad76a5f2b4a9ca525eb52075ce7e2; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676087708599; access_time=1676087707785; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-11 03:55:10,620 INFO [Listener at localhost/46205] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testGetAfterCompaction/dd87073924e61ec006a45079cb50294f/f1/726e888f51fa43119551a427b0a3e8be; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676087709427; access_time=1676087708612; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-11 03:55:10,620 INFO [Listener at localhost/46205] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testGetAfterCompaction/dd87073924e61ec006a45079cb50294f/f1/a24e770467ae446b921aa8dfce808122_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1676087710264; access_time=1676087709456; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-02-11 03:55:10,620 INFO [Listener at localhost/46205] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testGetAfterCompaction/dd87073924e61ec006a45079cb50294f/f1/eb2ad76a5f2b4a9ca525eb52075ce7e2; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676087708599; access_time=1676087707785; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-11 03:55:10,671 INFO [Listener at localhost/46205] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testGetAfterCompaction/dd87073924e61ec006a45079cb50294f/f1/726e888f51fa43119551a427b0a3e8be; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676087709427; access_time=1676087708612; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-11 03:55:10,671 INFO [Listener at localhost/46205] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testGetAfterCompaction/dd87073924e61ec006a45079cb50294f/f1/a24e770467ae446b921aa8dfce808122_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1676087710264; access_time=1676087709456; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-02-11 03:55:10,671 INFO [Listener at localhost/46205] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testGetAfterCompaction/dd87073924e61ec006a45079cb50294f/f1/eb2ad76a5f2b4a9ca525eb52075ce7e2; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676087708599; access_time=1676087707785; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-11 03:55:10,722 INFO [Listener at localhost/46205] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testGetAfterCompaction/dd87073924e61ec006a45079cb50294f/f1/726e888f51fa43119551a427b0a3e8be; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676087709427; access_time=1676087708612; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-11 03:55:10,722 INFO [Listener at localhost/46205] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testGetAfterCompaction/dd87073924e61ec006a45079cb50294f/f1/a24e770467ae446b921aa8dfce808122_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1676087710264; access_time=1676087709456; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-02-11 03:55:10,722 INFO [Listener at localhost/46205] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testGetAfterCompaction/dd87073924e61ec006a45079cb50294f/f1/eb2ad76a5f2b4a9ca525eb52075ce7e2; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676087708599; access_time=1676087707785; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-11 03:55:10,773 INFO [Listener at localhost/46205] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testGetAfterCompaction/dd87073924e61ec006a45079cb50294f/f1/726e888f51fa43119551a427b0a3e8be; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676087709427; access_time=1676087708612; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-11 03:55:10,773 INFO [Listener at localhost/46205] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testGetAfterCompaction/dd87073924e61ec006a45079cb50294f/f1/a24e770467ae446b921aa8dfce808122_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1676087710264; access_time=1676087709456; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-02-11 03:55:10,773 INFO [Listener at localhost/46205] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testGetAfterCompaction/dd87073924e61ec006a45079cb50294f/f1/eb2ad76a5f2b4a9ca525eb52075ce7e2; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676087708599; access_time=1676087707785; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-11 03:55:10,824 INFO [Listener at localhost/46205] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testGetAfterCompaction/dd87073924e61ec006a45079cb50294f/f1/726e888f51fa43119551a427b0a3e8be; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676087709427; access_time=1676087708612; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-11 03:55:10,824 INFO [Listener at localhost/46205] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testGetAfterCompaction/dd87073924e61ec006a45079cb50294f/f1/a24e770467ae446b921aa8dfce808122_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1676087710264; access_time=1676087709456; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-02-11 03:55:10,824 INFO [Listener at localhost/46205] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testGetAfterCompaction/dd87073924e61ec006a45079cb50294f/f1/eb2ad76a5f2b4a9ca525eb52075ce7e2; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676087708599; access_time=1676087707785; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-11 03:55:10,875 INFO [Listener at localhost/46205] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testGetAfterCompaction/dd87073924e61ec006a45079cb50294f/f1/726e888f51fa43119551a427b0a3e8be; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676087709427; access_time=1676087708612; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-11 03:55:10,875 INFO [Listener at localhost/46205] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testGetAfterCompaction/dd87073924e61ec006a45079cb50294f/f1/a24e770467ae446b921aa8dfce808122_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1676087710264; access_time=1676087709456; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-02-11 03:55:10,875 INFO [Listener at localhost/46205] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testGetAfterCompaction/dd87073924e61ec006a45079cb50294f/f1/eb2ad76a5f2b4a9ca525eb52075ce7e2; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676087708599; access_time=1676087707785; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-11 03:55:10,926 INFO [Listener at localhost/46205] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testGetAfterCompaction/dd87073924e61ec006a45079cb50294f/f1/726e888f51fa43119551a427b0a3e8be; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676087709427; access_time=1676087708612; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-11 03:55:10,926 INFO [Listener at localhost/46205] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testGetAfterCompaction/dd87073924e61ec006a45079cb50294f/f1/a24e770467ae446b921aa8dfce808122_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1676087710264; access_time=1676087709456; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-02-11 03:55:10,926 INFO [Listener at localhost/46205] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testGetAfterCompaction/dd87073924e61ec006a45079cb50294f/f1/eb2ad76a5f2b4a9ca525eb52075ce7e2; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676087708599; access_time=1676087707785; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-11 03:55:10,977 INFO [Listener at localhost/46205] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testGetAfterCompaction/dd87073924e61ec006a45079cb50294f/f1/726e888f51fa43119551a427b0a3e8be; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676087709427; access_time=1676087708612; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-11 03:55:10,977 INFO [Listener at localhost/46205] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testGetAfterCompaction/dd87073924e61ec006a45079cb50294f/f1/a24e770467ae446b921aa8dfce808122_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1676087710264; access_time=1676087709456; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-02-11 03:55:10,977 INFO [Listener at localhost/46205] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testGetAfterCompaction/dd87073924e61ec006a45079cb50294f/f1/eb2ad76a5f2b4a9ca525eb52075ce7e2; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676087708599; access_time=1676087707785; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-11 03:55:11,027 INFO [Listener at localhost/46205] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testGetAfterCompaction/dd87073924e61ec006a45079cb50294f/f1/726e888f51fa43119551a427b0a3e8be; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676087709427; access_time=1676087708612; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-11 03:55:11,028 INFO [Listener at localhost/46205] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testGetAfterCompaction/dd87073924e61ec006a45079cb50294f/f1/a24e770467ae446b921aa8dfce808122_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1676087710264; access_time=1676087709456; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-02-11 03:55:11,028 INFO [Listener at localhost/46205] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testGetAfterCompaction/dd87073924e61ec006a45079cb50294f/f1/eb2ad76a5f2b4a9ca525eb52075ce7e2; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676087708599; access_time=1676087707785; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-11 03:55:11,078 INFO [Listener at localhost/46205] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testGetAfterCompaction/dd87073924e61ec006a45079cb50294f/f1/726e888f51fa43119551a427b0a3e8be; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676087709427; access_time=1676087708612; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-11 03:55:11,078 INFO [Listener at localhost/46205] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testGetAfterCompaction/dd87073924e61ec006a45079cb50294f/f1/a24e770467ae446b921aa8dfce808122_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1676087710264; access_time=1676087709456; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-02-11 03:55:11,078 INFO [Listener at localhost/46205] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testGetAfterCompaction/dd87073924e61ec006a45079cb50294f/f1/eb2ad76a5f2b4a9ca525eb52075ce7e2; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676087708599; access_time=1676087707785; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-11 03:55:11,129 INFO [Listener at localhost/46205] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testGetAfterCompaction/dd87073924e61ec006a45079cb50294f/f1/726e888f51fa43119551a427b0a3e8be; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676087709427; access_time=1676087708612; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-11 03:55:11,129 INFO [Listener at localhost/46205] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testGetAfterCompaction/dd87073924e61ec006a45079cb50294f/f1/a24e770467ae446b921aa8dfce808122_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1676087710264; access_time=1676087709456; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-02-11 03:55:11,129 INFO [Listener at localhost/46205] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testGetAfterCompaction/dd87073924e61ec006a45079cb50294f/f1/eb2ad76a5f2b4a9ca525eb52075ce7e2; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676087708599; access_time=1676087707785; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-11 03:55:11,139 DEBUG [RS:0;jenkins-hbase4:37011-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testGetAfterCompaction/dd87073924e61ec006a45079cb50294f/.tmp/f1/d154cdc36f054a47a8de6baa3dc0d87e as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testGetAfterCompaction/dd87073924e61ec006a45079cb50294f/f1/d154cdc36f054a47a8de6baa3dc0d87e 2023-02-11 03:55:11,144 INFO [RS:0;jenkins-hbase4:37011-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 2 file(s) in dd87073924e61ec006a45079cb50294f/f1 of dd87073924e61ec006a45079cb50294f into d154cdc36f054a47a8de6baa3dc0d87e(size=5.1 K), total size for store is 10.0 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-11 03:55:11,144 DEBUG [RS:0;jenkins-hbase4:37011-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for dd87073924e61ec006a45079cb50294f: 2023-02-11 03:55:11,144 INFO [RS:0;jenkins-hbase4:37011-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testGetAfterCompaction,,1676087705666.dd87073924e61ec006a45079cb50294f., storeName=f1, fileCount=2, fileSize=10.0 K (4.9 K, 5.1 K), priority=2, time=1676087710310; duration=0sec 2023-02-11 03:55:11,144 DEBUG [RS:0;jenkins-hbase4:37011-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-11 03:55:11,180 INFO [Listener at localhost/46205] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testGetAfterCompaction/dd87073924e61ec006a45079cb50294f/f1/726e888f51fa43119551a427b0a3e8be; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676087709427; access_time=1676087708612; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-11 03:55:11,180 INFO [Listener at localhost/46205] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testGetAfterCompaction/dd87073924e61ec006a45079cb50294f/f1/a24e770467ae446b921aa8dfce808122_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1676087710264; access_time=1676087709456; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-02-11 03:55:11,180 INFO [Listener at localhost/46205] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testGetAfterCompaction/dd87073924e61ec006a45079cb50294f/f1/d154cdc36f054a47a8de6baa3dc0d87e; isDirectory=false; length=5228; replication=1; blocksize=134217728; modification_time=1676087711135; access_time=1676087710319; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-11 03:55:11,180 INFO [Listener at localhost/46205] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testGetAfterCompaction/dd87073924e61ec006a45079cb50294f/f1/eb2ad76a5f2b4a9ca525eb52075ce7e2; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676087708599; access_time=1676087707785; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-11 03:55:11,191 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testGetAfterCompaction/dd87073924e61ec006a45079cb50294f/f1/726e888f51fa43119551a427b0a3e8be, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testGetAfterCompaction/dd87073924e61ec006a45079cb50294f/f1/a24e770467ae446b921aa8dfce808122_SeqId_12_] to archive 2023-02-11 03:55:11,191 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-11 03:55:11,193 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testGetAfterCompaction/dd87073924e61ec006a45079cb50294f/f1/726e888f51fa43119551a427b0a3e8be to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testGetAfterCompaction/dd87073924e61ec006a45079cb50294f/f1/726e888f51fa43119551a427b0a3e8be 2023-02-11 03:55:11,193 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testGetAfterCompaction/dd87073924e61ec006a45079cb50294f/f1/a24e770467ae446b921aa8dfce808122_SeqId_12_ to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testGetAfterCompaction/dd87073924e61ec006a45079cb50294f/f1/a24e770467ae446b921aa8dfce808122_SeqId_12_ 2023-02-11 03:55:11,231 INFO [Listener at localhost/46205] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testGetAfterCompaction/dd87073924e61ec006a45079cb50294f/f1/d154cdc36f054a47a8de6baa3dc0d87e; isDirectory=false; length=5228; replication=1; blocksize=134217728; modification_time=1676087711135; access_time=1676087710319; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-11 03:55:11,231 INFO [Listener at localhost/46205] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testGetAfterCompaction/dd87073924e61ec006a45079cb50294f/f1/eb2ad76a5f2b4a9ca525eb52075ce7e2; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676087708599; access_time=1676087707785; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-11 03:55:11,349 INFO [Listener at localhost/46205] client.HBaseAdmin$15(890): Started disable of testGetAfterCompaction 2023-02-11 03:55:11,354 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] master.HMaster$11(2405): Client=jenkins//172.31.14.131 disable testGetAfterCompaction 2023-02-11 03:55:11,359 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] procedure2.ProcedureExecutor(1029): Stored pid=75, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testGetAfterCompaction 2023-02-11 03:55:11,362 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testGetAfterCompaction","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676087711362"}]},"ts":"1676087711362"} 2023-02-11 03:55:11,364 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=testGetAfterCompaction, state=DISABLING in hbase:meta 2023-02-11 03:55:11,364 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] master.MasterRpcServices(1227): Checking to see if procedure is done pid=75 2023-02-11 03:55:11,368 INFO [PEWorker-3] procedure.DisableTableProcedure(293): Set testGetAfterCompaction to state=DISABLING 2023-02-11 03:55:11,369 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=76, ppid=75, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testGetAfterCompaction, region=dd87073924e61ec006a45079cb50294f, UNASSIGN}] 2023-02-11 03:55:11,370 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=76, ppid=75, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testGetAfterCompaction, region=dd87073924e61ec006a45079cb50294f, UNASSIGN 2023-02-11 03:55:11,371 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=76 updating hbase:meta row=dd87073924e61ec006a45079cb50294f, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:55:11,371 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testGetAfterCompaction,,1676087705666.dd87073924e61ec006a45079cb50294f.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676087711371"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676087711371"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676087711371"}]},"ts":"1676087711371"} 2023-02-11 03:55:11,372 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=77, ppid=76, state=RUNNABLE; CloseRegionProcedure dd87073924e61ec006a45079cb50294f, server=jenkins-hbase4.apache.org,37011,1676087618428}] 2023-02-11 03:55:11,465 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] master.MasterRpcServices(1227): Checking to see if procedure is done pid=75 2023-02-11 03:55:11,524 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close dd87073924e61ec006a45079cb50294f 2023-02-11 03:55:11,524 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing dd87073924e61ec006a45079cb50294f, disabling compactions & flushes 2023-02-11 03:55:11,524 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testGetAfterCompaction,,1676087705666.dd87073924e61ec006a45079cb50294f. 2023-02-11 03:55:11,524 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testGetAfterCompaction,,1676087705666.dd87073924e61ec006a45079cb50294f. 2023-02-11 03:55:11,524 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testGetAfterCompaction,,1676087705666.dd87073924e61ec006a45079cb50294f. after waiting 0 ms 2023-02-11 03:55:11,524 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testGetAfterCompaction,,1676087705666.dd87073924e61ec006a45079cb50294f. 2023-02-11 03:55:11,528 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testGetAfterCompaction/dd87073924e61ec006a45079cb50294f/recovered.edits/15.seqid, newMaxSeqId=15, maxSeqId=1 2023-02-11 03:55:11,528 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.mob.compactions.TestMobCompactor$CompactTwoLatestHfilesCopro 2023-02-11 03:55:11,529 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testGetAfterCompaction,,1676087705666.dd87073924e61ec006a45079cb50294f. 2023-02-11 03:55:11,529 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for dd87073924e61ec006a45079cb50294f: Waiting for close lock at 1676087711524Running coprocessor pre-close hooks at 1676087711524Disabling compacts and flushes for region at 1676087711524Disabling writes for close at 1676087711524Writing region close event to WAL at 1676087711525 (+1 ms)Running coprocessor post-close hooks at 1676087711528 (+3 ms)Closed at 1676087711529 (+1 ms) 2023-02-11 03:55:11,531 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed dd87073924e61ec006a45079cb50294f 2023-02-11 03:55:11,531 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=76 updating hbase:meta row=dd87073924e61ec006a45079cb50294f, regionState=CLOSED 2023-02-11 03:55:11,531 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testGetAfterCompaction,,1676087705666.dd87073924e61ec006a45079cb50294f.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676087711531"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676087711531"}]},"ts":"1676087711531"} 2023-02-11 03:55:11,533 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=77, resume processing ppid=76 2023-02-11 03:55:11,533 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=77, ppid=76, state=SUCCESS; CloseRegionProcedure dd87073924e61ec006a45079cb50294f, server=jenkins-hbase4.apache.org,37011,1676087618428 in 160 msec 2023-02-11 03:55:11,534 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=76, resume processing ppid=75 2023-02-11 03:55:11,534 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=76, ppid=75, state=SUCCESS; TransitRegionStateProcedure table=testGetAfterCompaction, region=dd87073924e61ec006a45079cb50294f, UNASSIGN in 164 msec 2023-02-11 03:55:11,535 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testGetAfterCompaction","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676087711535"}]},"ts":"1676087711535"} 2023-02-11 03:55:11,536 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=testGetAfterCompaction, state=DISABLED in hbase:meta 2023-02-11 03:55:11,537 INFO [PEWorker-3] procedure.DisableTableProcedure(305): Set testGetAfterCompaction to state=DISABLED 2023-02-11 03:55:11,538 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=75, state=SUCCESS; DisableTableProcedure table=testGetAfterCompaction in 182 msec 2023-02-11 03:55:11,666 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] master.MasterRpcServices(1227): Checking to see if procedure is done pid=75 2023-02-11 03:55:11,666 INFO [Listener at localhost/46205] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testGetAfterCompaction, procId: 75 completed 2023-02-11 03:55:11,670 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] master.HMaster$5(2215): Client=jenkins//172.31.14.131 delete testGetAfterCompaction 2023-02-11 03:55:11,676 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] procedure2.ProcedureExecutor(1029): Stored pid=78, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testGetAfterCompaction 2023-02-11 03:55:11,677 DEBUG [PEWorker-5] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=78, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testGetAfterCompaction 2023-02-11 03:55:11,679 DEBUG [PEWorker-5] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=78, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testGetAfterCompaction 2023-02-11 03:55:11,680 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] master.MasterRpcServices(1227): Checking to see if procedure is done pid=78 2023-02-11 03:55:11,682 DEBUG [HFileArchiver-12] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/.tmp/data/default/testGetAfterCompaction/dd87073924e61ec006a45079cb50294f 2023-02-11 03:55:11,684 DEBUG [HFileArchiver-12] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/.tmp/data/default/testGetAfterCompaction/dd87073924e61ec006a45079cb50294f/f1, FileablePath, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/.tmp/data/default/testGetAfterCompaction/dd87073924e61ec006a45079cb50294f/recovered.edits] 2023-02-11 03:55:11,687 DEBUG [HFileArchiver-12] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/.tmp/data/default/testGetAfterCompaction/dd87073924e61ec006a45079cb50294f/f1/d154cdc36f054a47a8de6baa3dc0d87e to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testGetAfterCompaction/dd87073924e61ec006a45079cb50294f/f1/d154cdc36f054a47a8de6baa3dc0d87e 2023-02-11 03:55:11,688 DEBUG [HFileArchiver-12] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/.tmp/data/default/testGetAfterCompaction/dd87073924e61ec006a45079cb50294f/f1/eb2ad76a5f2b4a9ca525eb52075ce7e2 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testGetAfterCompaction/dd87073924e61ec006a45079cb50294f/f1/eb2ad76a5f2b4a9ca525eb52075ce7e2 2023-02-11 03:55:11,691 DEBUG [HFileArchiver-12] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/.tmp/data/default/testGetAfterCompaction/dd87073924e61ec006a45079cb50294f/recovered.edits/15.seqid to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testGetAfterCompaction/dd87073924e61ec006a45079cb50294f/recovered.edits/15.seqid 2023-02-11 03:55:11,691 DEBUG [HFileArchiver-12] backup.HFileArchiver(596): Deleted hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/.tmp/data/default/testGetAfterCompaction/dd87073924e61ec006a45079cb50294f 2023-02-11 03:55:11,691 DEBUG [PEWorker-5] procedure.DeleteTableProcedure(328): Archived testGetAfterCompaction regions 2023-02-11 03:55:11,691 DEBUG [PEWorker-5] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0 2023-02-11 03:55:11,692 DEBUG [PEWorker-5] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1] 2023-02-11 03:55:11,694 DEBUG [PEWorker-5] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e202302114242fde04ccf43e99cbffd21094abe57 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e202302114242fde04ccf43e99cbffd21094abe57 2023-02-11 03:55:11,695 DEBUG [PEWorker-5] backup.HFileArchiver(596): Deleted hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0 2023-02-11 03:55:11,697 DEBUG [PEWorker-5] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=78, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testGetAfterCompaction 2023-02-11 03:55:11,717 WARN [PEWorker-5] procedure.DeleteTableProcedure(384): Deleting some vestigial 1 rows of testGetAfterCompaction from hbase:meta 2023-02-11 03:55:11,719 DEBUG [PEWorker-5] procedure.DeleteTableProcedure(421): Removing 'testGetAfterCompaction' descriptor. 2023-02-11 03:55:11,720 DEBUG [PEWorker-5] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=78, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testGetAfterCompaction 2023-02-11 03:55:11,720 DEBUG [PEWorker-5] procedure.DeleteTableProcedure(411): Removing 'testGetAfterCompaction' from region states. 2023-02-11 03:55:11,720 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testGetAfterCompaction,,1676087705666.dd87073924e61ec006a45079cb50294f.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676087711720"}]},"ts":"9223372036854775807"} 2023-02-11 03:55:11,722 INFO [PEWorker-5] hbase.MetaTableAccessor(1788): Deleted 1 regions from META 2023-02-11 03:55:11,722 DEBUG [PEWorker-5] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => dd87073924e61ec006a45079cb50294f, NAME => 'testGetAfterCompaction,,1676087705666.dd87073924e61ec006a45079cb50294f.', STARTKEY => '', ENDKEY => ''}] 2023-02-11 03:55:11,722 DEBUG [PEWorker-5] procedure.DeleteTableProcedure(415): Marking 'testGetAfterCompaction' as deleted. 2023-02-11 03:55:11,722 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testGetAfterCompaction","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1676087711722"}]},"ts":"9223372036854775807"} 2023-02-11 03:55:11,723 INFO [PEWorker-5] hbase.MetaTableAccessor(1658): Deleted table testGetAfterCompaction state from META 2023-02-11 03:55:11,725 DEBUG [PEWorker-5] procedure.DeleteTableProcedure(130): Finished pid=78, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testGetAfterCompaction 2023-02-11 03:55:11,726 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=78, state=SUCCESS; DeleteTableProcedure table=testGetAfterCompaction in 54 msec 2023-02-11 03:55:11,781 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] master.MasterRpcServices(1227): Checking to see if procedure is done pid=78 2023-02-11 03:55:11,781 INFO [Listener at localhost/46205] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testGetAfterCompaction, procId: 78 completed 2023-02-11 03:55:11,793 INFO [Listener at localhost/46205] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testGetAfterCompaction Thread=235 (was 249), OpenFileDescriptor=589 (was 640), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=156 (was 175), ProcessCount=168 (was 168), AvailableMemoryMB=3093 (was 3102) 2023-02-11 03:55:11,804 INFO [Listener at localhost/46205] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testCompactionWithHFileLink Thread=235, OpenFileDescriptor=589, MaxFileDescriptor=60000, SystemLoadAverage=156, ProcessCount=168, AvailableMemoryMB=3092 2023-02-11 03:55:11,806 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] master.HMaster$4(2102): Client=jenkins//172.31.14.131 create 'testCompactionWithHFileLink', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} 2023-02-11 03:55:11,807 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] procedure2.ProcedureExecutor(1029): Stored pid=79, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testCompactionWithHFileLink 2023-02-11 03:55:11,808 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=79, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testCompactionWithHFileLink execute state=CREATE_TABLE_PRE_OPERATION 2023-02-11 03:55:11,808 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] master.MasterRpcServices(697): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testCompactionWithHFileLink" procId is: 79 2023-02-11 03:55:11,809 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=79, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testCompactionWithHFileLink execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-11 03:55:11,809 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] master.MasterRpcServices(1227): Checking to see if procedure is done pid=79 2023-02-11 03:55:11,811 DEBUG [HFileArchiver-13] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/.tmp/data/default/testCompactionWithHFileLink/186d87f306418fb25d295d824c1272fe 2023-02-11 03:55:11,811 DEBUG [HFileArchiver-14] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/.tmp/data/default/testCompactionWithHFileLink/ea4735f297efbbfcc4cbc0bd3d6e42e0 2023-02-11 03:55:11,811 DEBUG [HFileArchiver-15] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/.tmp/data/default/testCompactionWithHFileLink/621cd59809a16fd910172a9890d26263 2023-02-11 03:55:11,811 DEBUG [HFileArchiver-13] backup.HFileArchiver(153): Directory hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/.tmp/data/default/testCompactionWithHFileLink/186d87f306418fb25d295d824c1272fe empty. 2023-02-11 03:55:11,811 DEBUG [HFileArchiver-14] backup.HFileArchiver(153): Directory hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/.tmp/data/default/testCompactionWithHFileLink/ea4735f297efbbfcc4cbc0bd3d6e42e0 empty. 2023-02-11 03:55:11,811 DEBUG [HFileArchiver-15] backup.HFileArchiver(153): Directory hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/.tmp/data/default/testCompactionWithHFileLink/621cd59809a16fd910172a9890d26263 empty. 2023-02-11 03:55:11,812 DEBUG [HFileArchiver-13] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/.tmp/data/default/testCompactionWithHFileLink/186d87f306418fb25d295d824c1272fe 2023-02-11 03:55:11,812 DEBUG [HFileArchiver-14] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/.tmp/data/default/testCompactionWithHFileLink/ea4735f297efbbfcc4cbc0bd3d6e42e0 2023-02-11 03:55:11,812 DEBUG [HFileArchiver-15] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/.tmp/data/default/testCompactionWithHFileLink/621cd59809a16fd910172a9890d26263 2023-02-11 03:55:11,812 DEBUG [PEWorker-2] procedure.DeleteTableProcedure(328): Archived testCompactionWithHFileLink regions 2023-02-11 03:55:11,910 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] master.MasterRpcServices(1227): Checking to see if procedure is done pid=79 2023-02-11 03:55:12,111 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] master.MasterRpcServices(1227): Checking to see if procedure is done pid=79 2023-02-11 03:55:12,219 DEBUG [PEWorker-2] util.FSTableDescriptors(570): Wrote into hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/.tmp/data/default/testCompactionWithHFileLink/.tabledesc/.tableinfo.0000000001 2023-02-11 03:55:12,220 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(7671): creating {ENCODED => 186d87f306418fb25d295d824c1272fe, NAME => 'testCompactionWithHFileLink,,1676087711806.186d87f306418fb25d295d824c1272fe.', STARTKEY => '', ENDKEY => '1'}, tableDescriptor='testCompactionWithHFileLink', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/.tmp 2023-02-11 03:55:12,220 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(7671): creating {ENCODED => 621cd59809a16fd910172a9890d26263, NAME => 'testCompactionWithHFileLink,2,1676087711806.621cd59809a16fd910172a9890d26263.', STARTKEY => '2', ENDKEY => ''}, tableDescriptor='testCompactionWithHFileLink', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/.tmp 2023-02-11 03:55:12,221 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(7671): creating {ENCODED => ea4735f297efbbfcc4cbc0bd3d6e42e0, NAME => 'testCompactionWithHFileLink,1,1676087711806.ea4735f297efbbfcc4cbc0bd3d6e42e0.', STARTKEY => '1', ENDKEY => '2'}, tableDescriptor='testCompactionWithHFileLink', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/.tmp 2023-02-11 03:55:12,228 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(865): Instantiated testCompactionWithHFileLink,2,1676087711806.621cd59809a16fd910172a9890d26263.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-11 03:55:12,228 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(865): Instantiated testCompactionWithHFileLink,,1676087711806.186d87f306418fb25d295d824c1272fe.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-11 03:55:12,229 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(1603): Closing 621cd59809a16fd910172a9890d26263, disabling compactions & flushes 2023-02-11 03:55:12,229 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(1603): Closing 186d87f306418fb25d295d824c1272fe, disabling compactions & flushes 2023-02-11 03:55:12,229 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(1625): Closing region testCompactionWithHFileLink,2,1676087711806.621cd59809a16fd910172a9890d26263. 2023-02-11 03:55:12,229 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(1625): Closing region testCompactionWithHFileLink,,1676087711806.186d87f306418fb25d295d824c1272fe. 2023-02-11 03:55:12,229 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(1646): Waiting without time limit for close lock on testCompactionWithHFileLink,2,1676087711806.621cd59809a16fd910172a9890d26263. 2023-02-11 03:55:12,229 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testCompactionWithHFileLink,,1676087711806.186d87f306418fb25d295d824c1272fe. 2023-02-11 03:55:12,229 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(1713): Acquired close lock on testCompactionWithHFileLink,2,1676087711806.621cd59809a16fd910172a9890d26263. after waiting 0 ms 2023-02-11 03:55:12,229 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(1723): Updates disabled for region testCompactionWithHFileLink,2,1676087711806.621cd59809a16fd910172a9890d26263. 2023-02-11 03:55:12,229 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(1713): Acquired close lock on testCompactionWithHFileLink,,1676087711806.186d87f306418fb25d295d824c1272fe. after waiting 0 ms 2023-02-11 03:55:12,229 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(1723): Updates disabled for region testCompactionWithHFileLink,,1676087711806.186d87f306418fb25d295d824c1272fe. 2023-02-11 03:55:12,229 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(1837): Closed testCompactionWithHFileLink,2,1676087711806.621cd59809a16fd910172a9890d26263. 2023-02-11 03:55:12,229 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(1837): Closed testCompactionWithHFileLink,,1676087711806.186d87f306418fb25d295d824c1272fe. 2023-02-11 03:55:12,229 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(1557): Region close journal for 621cd59809a16fd910172a9890d26263: Waiting for close lock at 1676087712229Disabling compacts and flushes for region at 1676087712229Disabling writes for close at 1676087712229Writing region close event to WAL at 1676087712229Closed at 1676087712229 2023-02-11 03:55:12,229 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(865): Instantiated testCompactionWithHFileLink,1,1676087711806.ea4735f297efbbfcc4cbc0bd3d6e42e0.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-11 03:55:12,229 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(1557): Region close journal for 186d87f306418fb25d295d824c1272fe: Waiting for close lock at 1676087712229Disabling compacts and flushes for region at 1676087712229Disabling writes for close at 1676087712229Writing region close event to WAL at 1676087712229Closed at 1676087712229 2023-02-11 03:55:12,229 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(1603): Closing ea4735f297efbbfcc4cbc0bd3d6e42e0, disabling compactions & flushes 2023-02-11 03:55:12,229 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(1625): Closing region testCompactionWithHFileLink,1,1676087711806.ea4735f297efbbfcc4cbc0bd3d6e42e0. 2023-02-11 03:55:12,229 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(1646): Waiting without time limit for close lock on testCompactionWithHFileLink,1,1676087711806.ea4735f297efbbfcc4cbc0bd3d6e42e0. 2023-02-11 03:55:12,229 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(1713): Acquired close lock on testCompactionWithHFileLink,1,1676087711806.ea4735f297efbbfcc4cbc0bd3d6e42e0. after waiting 0 ms 2023-02-11 03:55:12,229 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(1723): Updates disabled for region testCompactionWithHFileLink,1,1676087711806.ea4735f297efbbfcc4cbc0bd3d6e42e0. 2023-02-11 03:55:12,230 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(1837): Closed testCompactionWithHFileLink,1,1676087711806.ea4735f297efbbfcc4cbc0bd3d6e42e0. 2023-02-11 03:55:12,230 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(1557): Region close journal for ea4735f297efbbfcc4cbc0bd3d6e42e0: Waiting for close lock at 1676087712229Disabling compacts and flushes for region at 1676087712229Disabling writes for close at 1676087712229Writing region close event to WAL at 1676087712230 (+1 ms)Closed at 1676087712230 2023-02-11 03:55:12,231 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=79, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testCompactionWithHFileLink execute state=CREATE_TABLE_ADD_TO_META 2023-02-11 03:55:12,232 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,2,1676087711806.621cd59809a16fd910172a9890d26263.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1676087712232"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676087712232"}]},"ts":"1676087712232"} 2023-02-11 03:55:12,232 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,,1676087711806.186d87f306418fb25d295d824c1272fe.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1676087712232"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676087712232"}]},"ts":"1676087712232"} 2023-02-11 03:55:12,232 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,1,1676087711806.ea4735f297efbbfcc4cbc0bd3d6e42e0.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1676087712232"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676087712232"}]},"ts":"1676087712232"} 2023-02-11 03:55:12,234 INFO [PEWorker-2] hbase.MetaTableAccessor(1496): Added 3 regions to meta. 2023-02-11 03:55:12,235 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=79, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testCompactionWithHFileLink execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-11 03:55:12,235 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testCompactionWithHFileLink","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676087712235"}]},"ts":"1676087712235"} 2023-02-11 03:55:12,236 INFO [PEWorker-2] hbase.MetaTableAccessor(1635): Updated tableName=testCompactionWithHFileLink, state=ENABLING in hbase:meta 2023-02-11 03:55:12,241 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=80, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=186d87f306418fb25d295d824c1272fe, ASSIGN}, {pid=81, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=ea4735f297efbbfcc4cbc0bd3d6e42e0, ASSIGN}, {pid=82, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=621cd59809a16fd910172a9890d26263, ASSIGN}] 2023-02-11 03:55:12,242 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=80, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=186d87f306418fb25d295d824c1272fe, ASSIGN 2023-02-11 03:55:12,243 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=81, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=ea4735f297efbbfcc4cbc0bd3d6e42e0, ASSIGN 2023-02-11 03:55:12,243 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=82, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=621cd59809a16fd910172a9890d26263, ASSIGN 2023-02-11 03:55:12,243 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=80, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=186d87f306418fb25d295d824c1272fe, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,37011,1676087618428; forceNewPlan=false, retain=false 2023-02-11 03:55:12,243 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=82, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=621cd59809a16fd910172a9890d26263, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,37011,1676087618428; forceNewPlan=false, retain=false 2023-02-11 03:55:12,243 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=81, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=ea4735f297efbbfcc4cbc0bd3d6e42e0, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,37011,1676087618428; forceNewPlan=false, retain=false 2023-02-11 03:55:12,395 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=80 updating hbase:meta row=186d87f306418fb25d295d824c1272fe, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:55:12,395 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=81 updating hbase:meta row=ea4735f297efbbfcc4cbc0bd3d6e42e0, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:55:12,395 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=82 updating hbase:meta row=621cd59809a16fd910172a9890d26263, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:55:12,395 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,,1676087711806.186d87f306418fb25d295d824c1272fe.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1676087712395"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676087712395"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676087712395"}]},"ts":"1676087712395"} 2023-02-11 03:55:12,395 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,1,1676087711806.ea4735f297efbbfcc4cbc0bd3d6e42e0.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1676087712395"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676087712395"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676087712395"}]},"ts":"1676087712395"} 2023-02-11 03:55:12,395 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,2,1676087711806.621cd59809a16fd910172a9890d26263.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1676087712395"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676087712395"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676087712395"}]},"ts":"1676087712395"} 2023-02-11 03:55:12,397 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=83, ppid=80, state=RUNNABLE; OpenRegionProcedure 186d87f306418fb25d295d824c1272fe, server=jenkins-hbase4.apache.org,37011,1676087618428}] 2023-02-11 03:55:12,397 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=84, ppid=81, state=RUNNABLE; OpenRegionProcedure ea4735f297efbbfcc4cbc0bd3d6e42e0, server=jenkins-hbase4.apache.org,37011,1676087618428}] 2023-02-11 03:55:12,399 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=85, ppid=82, state=RUNNABLE; OpenRegionProcedure 621cd59809a16fd910172a9890d26263, server=jenkins-hbase4.apache.org,37011,1676087618428}] 2023-02-11 03:55:12,412 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] master.MasterRpcServices(1227): Checking to see if procedure is done pid=79 2023-02-11 03:55:12,552 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testCompactionWithHFileLink,,1676087711806.186d87f306418fb25d295d824c1272fe. 2023-02-11 03:55:12,552 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 186d87f306418fb25d295d824c1272fe, NAME => 'testCompactionWithHFileLink,,1676087711806.186d87f306418fb25d295d824c1272fe.', STARTKEY => '', ENDKEY => '1'} 2023-02-11 03:55:12,552 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testCompactionWithHFileLink 186d87f306418fb25d295d824c1272fe 2023-02-11 03:55:12,552 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testCompactionWithHFileLink,,1676087711806.186d87f306418fb25d295d824c1272fe.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-11 03:55:12,553 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 186d87f306418fb25d295d824c1272fe 2023-02-11 03:55:12,553 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 186d87f306418fb25d295d824c1272fe 2023-02-11 03:55:12,554 INFO [StoreOpener-186d87f306418fb25d295d824c1272fe-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 186d87f306418fb25d295d824c1272fe 2023-02-11 03:55:12,555 DEBUG [StoreOpener-186d87f306418fb25d295d824c1272fe-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/186d87f306418fb25d295d824c1272fe/family1 2023-02-11 03:55:12,555 DEBUG [StoreOpener-186d87f306418fb25d295d824c1272fe-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/186d87f306418fb25d295d824c1272fe/family1 2023-02-11 03:55:12,555 INFO [StoreOpener-186d87f306418fb25d295d824c1272fe-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 186d87f306418fb25d295d824c1272fe columnFamilyName family1 2023-02-11 03:55:12,556 INFO [StoreOpener-186d87f306418fb25d295d824c1272fe-1] regionserver.HStore(310): Store=186d87f306418fb25d295d824c1272fe/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 03:55:12,556 INFO [StoreOpener-186d87f306418fb25d295d824c1272fe-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 186d87f306418fb25d295d824c1272fe 2023-02-11 03:55:12,557 DEBUG [StoreOpener-186d87f306418fb25d295d824c1272fe-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/186d87f306418fb25d295d824c1272fe/family2 2023-02-11 03:55:12,557 DEBUG [StoreOpener-186d87f306418fb25d295d824c1272fe-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/186d87f306418fb25d295d824c1272fe/family2 2023-02-11 03:55:12,558 INFO [StoreOpener-186d87f306418fb25d295d824c1272fe-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 186d87f306418fb25d295d824c1272fe columnFamilyName family2 2023-02-11 03:55:12,559 INFO [StoreOpener-186d87f306418fb25d295d824c1272fe-1] regionserver.HStore(310): Store=186d87f306418fb25d295d824c1272fe/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 03:55:12,559 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/186d87f306418fb25d295d824c1272fe 2023-02-11 03:55:12,560 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/186d87f306418fb25d295d824c1272fe 2023-02-11 03:55:12,561 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testCompactionWithHFileLink descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-11 03:55:12,562 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 186d87f306418fb25d295d824c1272fe 2023-02-11 03:55:12,563 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/186d87f306418fb25d295d824c1272fe/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-11 03:55:12,564 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 186d87f306418fb25d295d824c1272fe; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=60847564, jitterRate=-0.09330064058303833}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-11 03:55:12,564 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 186d87f306418fb25d295d824c1272fe: Running coprocessor pre-open hook at 1676087712553Writing region info on filesystem at 1676087712553Initializing all the Stores at 1676087712553Instantiating store for column family {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1676087712553Instantiating store for column family {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1676087712553Cleaning up temporary data from old regions at 1676087712560 (+7 ms)Cleaning up detritus from prior splits at 1676087712560Running coprocessor post-open hooks at 1676087712564 (+4 ms)Region opened successfully at 1676087712564 2023-02-11 03:55:12,564 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testCompactionWithHFileLink,,1676087711806.186d87f306418fb25d295d824c1272fe., pid=83, masterSystemTime=1676087712549 2023-02-11 03:55:12,566 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testCompactionWithHFileLink,,1676087711806.186d87f306418fb25d295d824c1272fe. 2023-02-11 03:55:12,566 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testCompactionWithHFileLink,,1676087711806.186d87f306418fb25d295d824c1272fe. 2023-02-11 03:55:12,566 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testCompactionWithHFileLink,1,1676087711806.ea4735f297efbbfcc4cbc0bd3d6e42e0. 2023-02-11 03:55:12,566 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => ea4735f297efbbfcc4cbc0bd3d6e42e0, NAME => 'testCompactionWithHFileLink,1,1676087711806.ea4735f297efbbfcc4cbc0bd3d6e42e0.', STARTKEY => '1', ENDKEY => '2'} 2023-02-11 03:55:12,566 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testCompactionWithHFileLink ea4735f297efbbfcc4cbc0bd3d6e42e0 2023-02-11 03:55:12,566 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=80 updating hbase:meta row=186d87f306418fb25d295d824c1272fe, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:55:12,566 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testCompactionWithHFileLink,1,1676087711806.ea4735f297efbbfcc4cbc0bd3d6e42e0.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-11 03:55:12,566 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for ea4735f297efbbfcc4cbc0bd3d6e42e0 2023-02-11 03:55:12,566 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for ea4735f297efbbfcc4cbc0bd3d6e42e0 2023-02-11 03:55:12,566 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testCompactionWithHFileLink,,1676087711806.186d87f306418fb25d295d824c1272fe.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1676087712566"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676087712566"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676087712566"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676087712566"}]},"ts":"1676087712566"} 2023-02-11 03:55:12,567 INFO [StoreOpener-ea4735f297efbbfcc4cbc0bd3d6e42e0-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region ea4735f297efbbfcc4cbc0bd3d6e42e0 2023-02-11 03:55:12,569 DEBUG [StoreOpener-ea4735f297efbbfcc4cbc0bd3d6e42e0-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/ea4735f297efbbfcc4cbc0bd3d6e42e0/family1 2023-02-11 03:55:12,569 DEBUG [StoreOpener-ea4735f297efbbfcc4cbc0bd3d6e42e0-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/ea4735f297efbbfcc4cbc0bd3d6e42e0/family1 2023-02-11 03:55:12,569 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=83, resume processing ppid=80 2023-02-11 03:55:12,569 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=83, ppid=80, state=SUCCESS; OpenRegionProcedure 186d87f306418fb25d295d824c1272fe, server=jenkins-hbase4.apache.org,37011,1676087618428 in 171 msec 2023-02-11 03:55:12,569 INFO [StoreOpener-ea4735f297efbbfcc4cbc0bd3d6e42e0-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region ea4735f297efbbfcc4cbc0bd3d6e42e0 columnFamilyName family1 2023-02-11 03:55:12,570 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=80, ppid=79, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=186d87f306418fb25d295d824c1272fe, ASSIGN in 329 msec 2023-02-11 03:55:12,570 INFO [StoreOpener-ea4735f297efbbfcc4cbc0bd3d6e42e0-1] regionserver.HStore(310): Store=ea4735f297efbbfcc4cbc0bd3d6e42e0/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 03:55:12,570 INFO [StoreOpener-ea4735f297efbbfcc4cbc0bd3d6e42e0-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region ea4735f297efbbfcc4cbc0bd3d6e42e0 2023-02-11 03:55:12,571 DEBUG [StoreOpener-ea4735f297efbbfcc4cbc0bd3d6e42e0-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/ea4735f297efbbfcc4cbc0bd3d6e42e0/family2 2023-02-11 03:55:12,571 DEBUG [StoreOpener-ea4735f297efbbfcc4cbc0bd3d6e42e0-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/ea4735f297efbbfcc4cbc0bd3d6e42e0/family2 2023-02-11 03:55:12,571 INFO [StoreOpener-ea4735f297efbbfcc4cbc0bd3d6e42e0-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region ea4735f297efbbfcc4cbc0bd3d6e42e0 columnFamilyName family2 2023-02-11 03:55:12,572 INFO [StoreOpener-ea4735f297efbbfcc4cbc0bd3d6e42e0-1] regionserver.HStore(310): Store=ea4735f297efbbfcc4cbc0bd3d6e42e0/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 03:55:12,572 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/ea4735f297efbbfcc4cbc0bd3d6e42e0 2023-02-11 03:55:12,573 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/ea4735f297efbbfcc4cbc0bd3d6e42e0 2023-02-11 03:55:12,574 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testCompactionWithHFileLink descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-11 03:55:12,575 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for ea4735f297efbbfcc4cbc0bd3d6e42e0 2023-02-11 03:55:12,576 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/ea4735f297efbbfcc4cbc0bd3d6e42e0/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-11 03:55:12,577 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened ea4735f297efbbfcc4cbc0bd3d6e42e0; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=60634102, jitterRate=-0.09648147225379944}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-11 03:55:12,577 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for ea4735f297efbbfcc4cbc0bd3d6e42e0: Running coprocessor pre-open hook at 1676087712566Writing region info on filesystem at 1676087712566Initializing all the Stores at 1676087712567 (+1 ms)Instantiating store for column family {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1676087712567Instantiating store for column family {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1676087712567Cleaning up temporary data from old regions at 1676087712573 (+6 ms)Cleaning up detritus from prior splits at 1676087712573Running coprocessor post-open hooks at 1676087712577 (+4 ms)Region opened successfully at 1676087712577 2023-02-11 03:55:12,577 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testCompactionWithHFileLink,1,1676087711806.ea4735f297efbbfcc4cbc0bd3d6e42e0., pid=84, masterSystemTime=1676087712549 2023-02-11 03:55:12,578 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testCompactionWithHFileLink,1,1676087711806.ea4735f297efbbfcc4cbc0bd3d6e42e0. 2023-02-11 03:55:12,578 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testCompactionWithHFileLink,1,1676087711806.ea4735f297efbbfcc4cbc0bd3d6e42e0. 2023-02-11 03:55:12,579 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testCompactionWithHFileLink,2,1676087711806.621cd59809a16fd910172a9890d26263. 2023-02-11 03:55:12,579 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 621cd59809a16fd910172a9890d26263, NAME => 'testCompactionWithHFileLink,2,1676087711806.621cd59809a16fd910172a9890d26263.', STARTKEY => '2', ENDKEY => ''} 2023-02-11 03:55:12,579 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testCompactionWithHFileLink 621cd59809a16fd910172a9890d26263 2023-02-11 03:55:12,579 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=81 updating hbase:meta row=ea4735f297efbbfcc4cbc0bd3d6e42e0, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:55:12,579 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testCompactionWithHFileLink,2,1676087711806.621cd59809a16fd910172a9890d26263.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-11 03:55:12,579 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 621cd59809a16fd910172a9890d26263 2023-02-11 03:55:12,579 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 621cd59809a16fd910172a9890d26263 2023-02-11 03:55:12,579 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testCompactionWithHFileLink,1,1676087711806.ea4735f297efbbfcc4cbc0bd3d6e42e0.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1676087712579"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676087712579"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676087712579"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676087712579"}]},"ts":"1676087712579"} 2023-02-11 03:55:12,580 INFO [StoreOpener-621cd59809a16fd910172a9890d26263-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 621cd59809a16fd910172a9890d26263 2023-02-11 03:55:12,582 DEBUG [StoreOpener-621cd59809a16fd910172a9890d26263-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/621cd59809a16fd910172a9890d26263/family1 2023-02-11 03:55:12,582 DEBUG [StoreOpener-621cd59809a16fd910172a9890d26263-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/621cd59809a16fd910172a9890d26263/family1 2023-02-11 03:55:12,582 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=84, resume processing ppid=81 2023-02-11 03:55:12,582 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=84, ppid=81, state=SUCCESS; OpenRegionProcedure ea4735f297efbbfcc4cbc0bd3d6e42e0, server=jenkins-hbase4.apache.org,37011,1676087618428 in 183 msec 2023-02-11 03:55:12,582 INFO [StoreOpener-621cd59809a16fd910172a9890d26263-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 621cd59809a16fd910172a9890d26263 columnFamilyName family1 2023-02-11 03:55:12,583 INFO [StoreOpener-621cd59809a16fd910172a9890d26263-1] regionserver.HStore(310): Store=621cd59809a16fd910172a9890d26263/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 03:55:12,583 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=81, ppid=79, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=ea4735f297efbbfcc4cbc0bd3d6e42e0, ASSIGN in 342 msec 2023-02-11 03:55:12,583 INFO [StoreOpener-621cd59809a16fd910172a9890d26263-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 621cd59809a16fd910172a9890d26263 2023-02-11 03:55:12,584 DEBUG [StoreOpener-621cd59809a16fd910172a9890d26263-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/621cd59809a16fd910172a9890d26263/family2 2023-02-11 03:55:12,584 DEBUG [StoreOpener-621cd59809a16fd910172a9890d26263-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/621cd59809a16fd910172a9890d26263/family2 2023-02-11 03:55:12,584 INFO [StoreOpener-621cd59809a16fd910172a9890d26263-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 621cd59809a16fd910172a9890d26263 columnFamilyName family2 2023-02-11 03:55:12,585 INFO [StoreOpener-621cd59809a16fd910172a9890d26263-1] regionserver.HStore(310): Store=621cd59809a16fd910172a9890d26263/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 03:55:12,586 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/621cd59809a16fd910172a9890d26263 2023-02-11 03:55:12,586 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/621cd59809a16fd910172a9890d26263 2023-02-11 03:55:12,588 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testCompactionWithHFileLink descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-11 03:55:12,588 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 621cd59809a16fd910172a9890d26263 2023-02-11 03:55:12,590 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/621cd59809a16fd910172a9890d26263/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-11 03:55:12,591 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 621cd59809a16fd910172a9890d26263; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=59264106, jitterRate=-0.11689600348472595}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-11 03:55:12,591 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 621cd59809a16fd910172a9890d26263: Running coprocessor pre-open hook at 1676087712579Writing region info on filesystem at 1676087712579Initializing all the Stores at 1676087712580 (+1 ms)Instantiating store for column family {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1676087712580Instantiating store for column family {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1676087712580Cleaning up temporary data from old regions at 1676087712587 (+7 ms)Cleaning up detritus from prior splits at 1676087712587Running coprocessor post-open hooks at 1676087712591 (+4 ms)Region opened successfully at 1676087712591 2023-02-11 03:55:12,591 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testCompactionWithHFileLink,2,1676087711806.621cd59809a16fd910172a9890d26263., pid=85, masterSystemTime=1676087712549 2023-02-11 03:55:12,593 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testCompactionWithHFileLink,2,1676087711806.621cd59809a16fd910172a9890d26263. 2023-02-11 03:55:12,593 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testCompactionWithHFileLink,2,1676087711806.621cd59809a16fd910172a9890d26263. 2023-02-11 03:55:12,593 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=82 updating hbase:meta row=621cd59809a16fd910172a9890d26263, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:55:12,594 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testCompactionWithHFileLink,2,1676087711806.621cd59809a16fd910172a9890d26263.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1676087712593"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676087712593"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676087712593"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676087712593"}]},"ts":"1676087712593"} 2023-02-11 03:55:12,596 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=85, resume processing ppid=82 2023-02-11 03:55:12,596 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=85, ppid=82, state=SUCCESS; OpenRegionProcedure 621cd59809a16fd910172a9890d26263, server=jenkins-hbase4.apache.org,37011,1676087618428 in 197 msec 2023-02-11 03:55:12,597 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=82, resume processing ppid=79 2023-02-11 03:55:12,597 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=82, ppid=79, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=621cd59809a16fd910172a9890d26263, ASSIGN in 355 msec 2023-02-11 03:55:12,597 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=79, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testCompactionWithHFileLink execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-11 03:55:12,598 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testCompactionWithHFileLink","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676087712598"}]},"ts":"1676087712598"} 2023-02-11 03:55:12,599 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=testCompactionWithHFileLink, state=ENABLED in hbase:meta 2023-02-11 03:55:12,601 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=79, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testCompactionWithHFileLink execute state=CREATE_TABLE_POST_OPERATION 2023-02-11 03:55:12,601 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=79, state=SUCCESS; CreateTableProcedure table=testCompactionWithHFileLink in 795 msec 2023-02-11 03:55:12,913 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] master.MasterRpcServices(1227): Checking to see if procedure is done pid=79 2023-02-11 03:55:12,913 INFO [Listener at localhost/46205] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testCompactionWithHFileLink, procId: 79 completed 2023-02-11 03:55:12,920 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.HRegion(8801): writing data to region testCompactionWithHFileLink,,1676087711806.186d87f306418fb25d295d824c1272fe. with WAL disabled. Data may be lost in the event of a crash. 2023-02-11 03:55:12,921 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.HRegion(8801): writing data to region testCompactionWithHFileLink,1,1676087711806.ea4735f297efbbfcc4cbc0bd3d6e42e0. with WAL disabled. Data may be lost in the event of a crash. 2023-02-11 03:55:12,922 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.HRegion(8801): writing data to region testCompactionWithHFileLink,2,1676087711806.621cd59809a16fd910172a9890d26263. with WAL disabled. Data may be lost in the event of a crash. 2023-02-11 03:55:12,925 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 03:55:12,926 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] procedure.ProcedureCoordinator(165): Submitting procedure testCompactionWithHFileLink 2023-02-11 03:55:12,926 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testCompactionWithHFileLink' 2023-02-11 03:55:12,926 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:55:12,927 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testCompactionWithHFileLink' starting 'acquire' 2023-02-11 03:55:12,927 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testCompactionWithHFileLink', kicking off acquire phase on members. 2023-02-11 03:55:12,927 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-11 03:55:12,927 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-02-11 03:55:12,928 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:55:12,928 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:55:12,928 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:55:12,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:55:12,928 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:55:12,928 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 03:55:12,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-02-11 03:55:12,929 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-11 03:55:12,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 03:55:12,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-02-11 03:55:12,929 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testCompactionWithHFileLink 2023-02-11 03:55:12,929 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testCompactionWithHFileLink 2023-02-11 03:55:12,929 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testCompactionWithHFileLink' with timeout 60000ms 2023-02-11 03:55:12,929 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:55:12,929 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testCompactionWithHFileLink' starting 'acquire' stage 2023-02-11 03:55:12,930 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,2,1676087711806.621cd59809a16fd910172a9890d26263. 2023-02-11 03:55:12,930 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-02-11 03:55:12,930 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 03:55:12,930 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,2,1676087711806.621cd59809a16fd910172a9890d26263. started... 2023-02-11 03:55:12,930 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,,1676087711806.186d87f306418fb25d295d824c1272fe. 2023-02-11 03:55:12,930 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,,1676087711806.186d87f306418fb25d295d824c1272fe. started... 2023-02-11 03:55:12,930 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,1,1676087711806.ea4735f297efbbfcc4cbc0bd3d6e42e0. 2023-02-11 03:55:12,930 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 621cd59809a16fd910172a9890d26263 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-02-11 03:55:12,930 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,1,1676087711806.ea4735f297efbbfcc4cbc0bd3d6e42e0. started... 2023-02-11 03:55:12,931 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-1] regionserver.HRegion(2744): Flushing 186d87f306418fb25d295d824c1272fe 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-02-11 03:55:12,931 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-2] regionserver.HRegion(2744): Flushing ea4735f297efbbfcc4cbc0bd3d6e42e0 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-02-11 03:55:12,944 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202302116f5da94e2cf44f26952495b89148e481 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202302116f5da94e2cf44f26952495b89148e481 2023-02-11 03:55:13,345 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230211efe856603053418f95a0d3b45cd7d38d to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230211efe856603053418f95a0d3b45cd7d38d 2023-02-11 03:55:13,345 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230211bbcb00e25dbb444c96d85d72f4a0a09a to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230211bbcb00e25dbb444c96d85d72f4a0a09a 2023-02-11 03:55:13,349 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/621cd59809a16fd910172a9890d26263/.tmp/family1/a54c49bae3b34318920fc4197db7ae61 2023-02-11 03:55:13,350 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/ea4735f297efbbfcc4cbc0bd3d6e42e0/.tmp/family1/92743cf8525b4b5d877b358e975a3bec 2023-02-11 03:55:13,751 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/186d87f306418fb25d295d824c1272fe/.tmp/family1/2e6dcfa9c78543fcb2273f9d67b88e36 2023-02-11 03:55:13,764 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202302110d01c3b1b7ff473d89c5b36246154d46 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c202302110d01c3b1b7ff473d89c5b36246154d46 2023-02-11 03:55:13,771 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230211211c0620d3924f14baf3c085690d9d43 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230211211c0620d3924f14baf3c085690d9d43 2023-02-11 03:55:14,165 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230211cb55ca343ae8440a83b4999e2a1b2131 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230211cb55ca343ae8440a83b4999e2a1b2131 2023-02-11 03:55:14,170 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/621cd59809a16fd910172a9890d26263/.tmp/family2/774f5f8b4d2e463f983d817bdedf591e 2023-02-11 03:55:14,173 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/621cd59809a16fd910172a9890d26263/.tmp/family1/a54c49bae3b34318920fc4197db7ae61 as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/621cd59809a16fd910172a9890d26263/family1/a54c49bae3b34318920fc4197db7ae61 2023-02-11 03:55:14,175 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/ea4735f297efbbfcc4cbc0bd3d6e42e0/.tmp/family2/7141c179434b4a0f816573a88da1caf7 2023-02-11 03:55:14,177 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/621cd59809a16fd910172a9890d26263/family1/a54c49bae3b34318920fc4197db7ae61, entries=4, sequenceid=5, filesize=5.4 K 2023-02-11 03:55:14,177 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/621cd59809a16fd910172a9890d26263/.tmp/family2/774f5f8b4d2e463f983d817bdedf591e as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/621cd59809a16fd910172a9890d26263/family2/774f5f8b4d2e463f983d817bdedf591e 2023-02-11 03:55:14,179 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/ea4735f297efbbfcc4cbc0bd3d6e42e0/.tmp/family1/92743cf8525b4b5d877b358e975a3bec as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/ea4735f297efbbfcc4cbc0bd3d6e42e0/family1/92743cf8525b4b5d877b358e975a3bec 2023-02-11 03:55:14,181 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/621cd59809a16fd910172a9890d26263/family2/774f5f8b4d2e463f983d817bdedf591e, entries=2, sequenceid=5, filesize=5.1 K 2023-02-11 03:55:14,182 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for 621cd59809a16fd910172a9890d26263 in 1252ms, sequenceid=5, compaction requested=false 2023-02-11 03:55:14,182 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testCompactionWithHFileLink' 2023-02-11 03:55:14,182 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/ea4735f297efbbfcc4cbc0bd3d6e42e0/family1/92743cf8525b4b5d877b358e975a3bec, entries=4, sequenceid=5, filesize=5.4 K 2023-02-11 03:55:14,183 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 621cd59809a16fd910172a9890d26263: 2023-02-11 03:55:14,183 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,2,1676087711806.621cd59809a16fd910172a9890d26263. 2023-02-11 03:55:14,183 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-02-11 03:55:14,183 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/ea4735f297efbbfcc4cbc0bd3d6e42e0/.tmp/family2/7141c179434b4a0f816573a88da1caf7 as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/ea4735f297efbbfcc4cbc0bd3d6e42e0/family2/7141c179434b4a0f816573a88da1caf7 2023-02-11 03:55:14,187 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/ea4735f297efbbfcc4cbc0bd3d6e42e0/family2/7141c179434b4a0f816573a88da1caf7, entries=2, sequenceid=5, filesize=5.1 K 2023-02-11 03:55:14,187 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-2] regionserver.HRegion(2947): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for ea4735f297efbbfcc4cbc0bd3d6e42e0 in 1256ms, sequenceid=5, compaction requested=false 2023-02-11 03:55:14,188 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-2] regionserver.HRegion(2445): Flush status journal for ea4735f297efbbfcc4cbc0bd3d6e42e0: 2023-02-11 03:55:14,188 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,1,1676087711806.ea4735f297efbbfcc4cbc0bd3d6e42e0. 2023-02-11 03:55:14,188 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-02-11 03:55:14,570 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/186d87f306418fb25d295d824c1272fe/.tmp/family2/48173b765b124663a97b2f43b90e8a88 2023-02-11 03:55:14,574 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/186d87f306418fb25d295d824c1272fe/.tmp/family1/2e6dcfa9c78543fcb2273f9d67b88e36 as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/186d87f306418fb25d295d824c1272fe/family1/2e6dcfa9c78543fcb2273f9d67b88e36 2023-02-11 03:55:14,578 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/186d87f306418fb25d295d824c1272fe/family1/2e6dcfa9c78543fcb2273f9d67b88e36, entries=4, sequenceid=5, filesize=5.4 K 2023-02-11 03:55:14,579 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/186d87f306418fb25d295d824c1272fe/.tmp/family2/48173b765b124663a97b2f43b90e8a88 as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/186d87f306418fb25d295d824c1272fe/family2/48173b765b124663a97b2f43b90e8a88 2023-02-11 03:55:14,583 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/186d87f306418fb25d295d824c1272fe/family2/48173b765b124663a97b2f43b90e8a88, entries=2, sequenceid=5, filesize=5.1 K 2023-02-11 03:55:14,583 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-1] regionserver.HRegion(2947): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for 186d87f306418fb25d295d824c1272fe in 1652ms, sequenceid=5, compaction requested=false 2023-02-11 03:55:14,583 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-1] regionserver.HRegion(2445): Flush status journal for 186d87f306418fb25d295d824c1272fe: 2023-02-11 03:55:14,583 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,,1676087711806.186d87f306418fb25d295d824c1272fe. 2023-02-11 03:55:14,584 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-02-11 03:55:14,584 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-02-11 03:55:14,584 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:55:14,584 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testCompactionWithHFileLink' locally acquired 2023-02-11 03:55:14,584 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37011,1676087618428' joining acquired barrier for procedure (testCompactionWithHFileLink) in zk 2023-02-11 03:55:14,586 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-11 03:55:14,586 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:55:14,586 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:55:14,586 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:55:14,587 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:55:14,587 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-11 03:55:14,587 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testCompactionWithHFileLink' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 03:55:14,587 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:55:14,587 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:55:14,587 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-11 03:55:14,587 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:55:14,588 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:55:14,588 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37011,1676087618428' joining acquired barrier for procedure 'testCompactionWithHFileLink' on coordinator 2023-02-11 03:55:14,588 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@294d212c[Count = 0] remaining members to acquire global barrier 2023-02-11 03:55:14,588 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testCompactionWithHFileLink' starting 'in-barrier' execution. 2023-02-11 03:55:14,588 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-11 03:55:14,589 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-11 03:55:14,589 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-11 03:55:14,589 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-11 03:55:14,589 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testCompactionWithHFileLink' received 'reached' from coordinator. 2023-02-11 03:55:14,589 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:55:14,589 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 03:55:14,589 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testCompactionWithHFileLink' locally completed 2023-02-11 03:55:14,589 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testCompactionWithHFileLink' completed for member 'jenkins-hbase4.apache.org,37011,1676087618428' in zk 2023-02-11 03:55:14,590 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testCompactionWithHFileLink' has notified controller of completion 2023-02-11 03:55:14,590 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:55:14,590 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:55:14,590 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:55:14,591 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:55:14,591 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:55:14,591 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testCompactionWithHFileLink' completed. 2023-02-11 03:55:14,591 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:55:14,591 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:55:14,591 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-11 03:55:14,591 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:55:14,592 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:55:14,592 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-11 03:55:14,592 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:55:14,592 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testCompactionWithHFileLink' member 'jenkins-hbase4.apache.org,37011,1676087618428': 2023-02-11 03:55:14,592 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37011,1676087618428' released barrier for procedure'testCompactionWithHFileLink', counting down latch. Waiting for 0 more 2023-02-11 03:55:14,592 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testCompactionWithHFileLink' execution completed 2023-02-11 03:55:14,592 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 03:55:14,593 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 03:55:14,593 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testCompactionWithHFileLink 2023-02-11 03:55:14,593 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testCompactionWithHFileLinkincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 03:55:14,594 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-11 03:55:14,594 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-11 03:55:14,594 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-11 03:55:14,594 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:55:14,594 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:55:14,594 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-11 03:55:14,594 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:55:14,594 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-11 03:55:14,594 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:55:14,594 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:55:14,594 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:55:14,594 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:55:14,594 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-11 03:55:14,594 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-11 03:55:14,595 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:55:14,595 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-11 03:55:14,595 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:55:14,595 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:55:14,595 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:55:14,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-11 03:55:14,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:55:14,598 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:55:14,598 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:55:14,598 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-02-11 03:55:14,598 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:55:14,598 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testCompactionWithHFileLink' 2023-02-11 03:55:14,599 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 03:55:14,598 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:55:14,598 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-02-11 03:55:14,598 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:55:14,599 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:55:14,598 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:55:14,599 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-11 03:55:14,599 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testCompactionWithHFileLink'' to complete. (max 300000 ms per retry) 2023-02-11 03:55:14,599 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-11 03:55:14,599 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 03:55:14,599 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:55:14,599 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:55:14,599 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-11 03:55:14,699 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 03:55:14,700 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 03:55:14,701 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.HRegion(8801): writing data to region testCompactionWithHFileLink,,1676087711806.186d87f306418fb25d295d824c1272fe. with WAL disabled. Data may be lost in the event of a crash. 2023-02-11 03:55:14,701 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.HRegion(8801): writing data to region testCompactionWithHFileLink,1,1676087711806.ea4735f297efbbfcc4cbc0bd3d6e42e0. with WAL disabled. Data may be lost in the event of a crash. 2023-02-11 03:55:14,701 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.HRegion(8801): writing data to region testCompactionWithHFileLink,2,1676087711806.621cd59809a16fd910172a9890d26263. with WAL disabled. Data may be lost in the event of a crash. 2023-02-11 03:55:14,704 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 03:55:14,706 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] procedure.ProcedureCoordinator(143): Procedure testCompactionWithHFileLink was in running list but was completed. Accepting new attempt. 2023-02-11 03:55:14,706 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] procedure.ProcedureCoordinator(165): Submitting procedure testCompactionWithHFileLink 2023-02-11 03:55:14,706 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testCompactionWithHFileLink' 2023-02-11 03:55:14,707 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:55:14,707 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testCompactionWithHFileLink' starting 'acquire' 2023-02-11 03:55:14,707 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testCompactionWithHFileLink', kicking off acquire phase on members. 2023-02-11 03:55:14,707 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-11 03:55:14,707 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-02-11 03:55:14,708 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:55:14,708 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:55:14,708 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:55:14,708 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:55:14,709 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:55:14,709 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 03:55:14,709 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-02-11 03:55:14,709 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-11 03:55:14,709 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 03:55:14,709 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-02-11 03:55:14,710 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testCompactionWithHFileLink 2023-02-11 03:55:14,710 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testCompactionWithHFileLink is still present, removing 2023-02-11 03:55:14,710 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testCompactionWithHFileLink 2023-02-11 03:55:14,710 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testCompactionWithHFileLink' with timeout 60000ms 2023-02-11 03:55:14,710 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:55:14,710 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testCompactionWithHFileLink' starting 'acquire' stage 2023-02-11 03:55:14,710 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,2,1676087711806.621cd59809a16fd910172a9890d26263. 2023-02-11 03:55:14,710 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-02-11 03:55:14,710 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,1,1676087711806.ea4735f297efbbfcc4cbc0bd3d6e42e0. 2023-02-11 03:55:14,710 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,1,1676087711806.ea4735f297efbbfcc4cbc0bd3d6e42e0. started... 2023-02-11 03:55:14,710 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,,1676087711806.186d87f306418fb25d295d824c1272fe. 2023-02-11 03:55:14,710 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 03:55:14,710 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-2] regionserver.HRegion(2744): Flushing ea4735f297efbbfcc4cbc0bd3d6e42e0 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-02-11 03:55:14,710 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,2,1676087711806.621cd59809a16fd910172a9890d26263. started... 2023-02-11 03:55:14,710 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,,1676087711806.186d87f306418fb25d295d824c1272fe. started... 2023-02-11 03:55:14,711 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 621cd59809a16fd910172a9890d26263 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-02-11 03:55:14,711 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-1] regionserver.HRegion(2744): Flushing 186d87f306418fb25d295d824c1272fe 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-02-11 03:55:15,132 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230211e4a3d035f9844b869e82ec0923db93f1 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230211e4a3d035f9844b869e82ec0923db93f1 2023-02-11 03:55:15,137 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b2023021101c3e7c91909459eaa090b3ee73d4c03 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b2023021101c3e7c91909459eaa090b3ee73d4c03 2023-02-11 03:55:15,143 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230211464ca8637b744ea09bffcf8850a4d558 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230211464ca8637b744ea09bffcf8850a4d558 2023-02-11 03:55:15,537 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/186d87f306418fb25d295d824c1272fe/.tmp/family1/4e9479861f4540a6b8a67dee5f4b8c2c 2023-02-11 03:55:15,541 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/ea4735f297efbbfcc4cbc0bd3d6e42e0/.tmp/family1/0a642f51d6d94a79b4c5663961dee5c0 2023-02-11 03:55:15,548 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/621cd59809a16fd910172a9890d26263/.tmp/family1/051a49ec8101444f8e462520d66fa021 2023-02-11 03:55:15,951 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202302110f3eef6bbc07470d998a60ead6aad577 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e202302110f3eef6bbc07470d998a60ead6aad577 2023-02-11 03:55:15,954 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b2023021165313005ab884230948c8003bec6d4ef to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b2023021165313005ab884230948c8003bec6d4ef 2023-02-11 03:55:15,960 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202302117e5060c420684526926d62b7b980770c to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c202302117e5060c420684526926d62b7b980770c 2023-02-11 03:55:16,356 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/186d87f306418fb25d295d824c1272fe/.tmp/family2/039e3e0c12c742f29c5ea86a66e98c44 2023-02-11 03:55:16,360 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/ea4735f297efbbfcc4cbc0bd3d6e42e0/.tmp/family2/045314dc19574fe9979fbedc8e14acf8 2023-02-11 03:55:16,360 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/186d87f306418fb25d295d824c1272fe/.tmp/family1/4e9479861f4540a6b8a67dee5f4b8c2c as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/186d87f306418fb25d295d824c1272fe/family1/4e9479861f4540a6b8a67dee5f4b8c2c 2023-02-11 03:55:16,364 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/186d87f306418fb25d295d824c1272fe/family1/4e9479861f4540a6b8a67dee5f4b8c2c, entries=4, sequenceid=9, filesize=5.4 K 2023-02-11 03:55:16,364 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/ea4735f297efbbfcc4cbc0bd3d6e42e0/.tmp/family1/0a642f51d6d94a79b4c5663961dee5c0 as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/ea4735f297efbbfcc4cbc0bd3d6e42e0/family1/0a642f51d6d94a79b4c5663961dee5c0 2023-02-11 03:55:16,364 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/621cd59809a16fd910172a9890d26263/.tmp/family2/e000a979eed34f31ba1476e0d7c949da 2023-02-11 03:55:16,365 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/186d87f306418fb25d295d824c1272fe/.tmp/family2/039e3e0c12c742f29c5ea86a66e98c44 as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/186d87f306418fb25d295d824c1272fe/family2/039e3e0c12c742f29c5ea86a66e98c44 2023-02-11 03:55:16,368 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/ea4735f297efbbfcc4cbc0bd3d6e42e0/family1/0a642f51d6d94a79b4c5663961dee5c0, entries=4, sequenceid=9, filesize=5.4 K 2023-02-11 03:55:16,368 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/186d87f306418fb25d295d824c1272fe/family2/039e3e0c12c742f29c5ea86a66e98c44, entries=2, sequenceid=9, filesize=5.1 K 2023-02-11 03:55:16,369 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/621cd59809a16fd910172a9890d26263/.tmp/family1/051a49ec8101444f8e462520d66fa021 as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/621cd59809a16fd910172a9890d26263/family1/051a49ec8101444f8e462520d66fa021 2023-02-11 03:55:16,369 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/ea4735f297efbbfcc4cbc0bd3d6e42e0/.tmp/family2/045314dc19574fe9979fbedc8e14acf8 as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/ea4735f297efbbfcc4cbc0bd3d6e42e0/family2/045314dc19574fe9979fbedc8e14acf8 2023-02-11 03:55:16,369 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-1] regionserver.HRegion(2947): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for 186d87f306418fb25d295d824c1272fe in 1658ms, sequenceid=9, compaction requested=false 2023-02-11 03:55:16,369 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-1] regionserver.HRegion(2445): Flush status journal for 186d87f306418fb25d295d824c1272fe: 2023-02-11 03:55:16,369 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,,1676087711806.186d87f306418fb25d295d824c1272fe. 2023-02-11 03:55:16,369 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-02-11 03:55:16,372 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/621cd59809a16fd910172a9890d26263/family1/051a49ec8101444f8e462520d66fa021, entries=4, sequenceid=9, filesize=5.4 K 2023-02-11 03:55:16,373 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/ea4735f297efbbfcc4cbc0bd3d6e42e0/family2/045314dc19574fe9979fbedc8e14acf8, entries=2, sequenceid=9, filesize=5.1 K 2023-02-11 03:55:16,373 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/621cd59809a16fd910172a9890d26263/.tmp/family2/e000a979eed34f31ba1476e0d7c949da as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/621cd59809a16fd910172a9890d26263/family2/e000a979eed34f31ba1476e0d7c949da 2023-02-11 03:55:16,373 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-2] regionserver.HRegion(2947): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for ea4735f297efbbfcc4cbc0bd3d6e42e0 in 1663ms, sequenceid=9, compaction requested=false 2023-02-11 03:55:16,373 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-2] regionserver.HRegion(2445): Flush status journal for ea4735f297efbbfcc4cbc0bd3d6e42e0: 2023-02-11 03:55:16,373 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,1,1676087711806.ea4735f297efbbfcc4cbc0bd3d6e42e0. 2023-02-11 03:55:16,373 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-02-11 03:55:16,376 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/621cd59809a16fd910172a9890d26263/family2/e000a979eed34f31ba1476e0d7c949da, entries=2, sequenceid=9, filesize=5.1 K 2023-02-11 03:55:16,377 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for 621cd59809a16fd910172a9890d26263 in 1666ms, sequenceid=9, compaction requested=false 2023-02-11 03:55:16,377 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 621cd59809a16fd910172a9890d26263: 2023-02-11 03:55:16,377 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,2,1676087711806.621cd59809a16fd910172a9890d26263. 2023-02-11 03:55:16,377 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-02-11 03:55:16,377 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-02-11 03:55:16,377 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:55:16,377 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testCompactionWithHFileLink' locally acquired 2023-02-11 03:55:16,377 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37011,1676087618428' joining acquired barrier for procedure (testCompactionWithHFileLink) in zk 2023-02-11 03:55:16,378 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-11 03:55:16,378 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:55:16,379 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:55:16,379 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:55:16,379 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:55:16,379 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-11 03:55:16,379 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testCompactionWithHFileLink' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 03:55:16,379 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:55:16,379 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:55:16,379 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-11 03:55:16,380 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:55:16,380 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:55:16,380 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37011,1676087618428' joining acquired barrier for procedure 'testCompactionWithHFileLink' on coordinator 2023-02-11 03:55:16,380 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@25ec8eb4[Count = 0] remaining members to acquire global barrier 2023-02-11 03:55:16,380 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testCompactionWithHFileLink' starting 'in-barrier' execution. 2023-02-11 03:55:16,380 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-11 03:55:16,381 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-11 03:55:16,381 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-11 03:55:16,381 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-11 03:55:16,381 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testCompactionWithHFileLink' received 'reached' from coordinator. 2023-02-11 03:55:16,381 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testCompactionWithHFileLink' locally completed 2023-02-11 03:55:16,381 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:55:16,381 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 03:55:16,381 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testCompactionWithHFileLink' completed for member 'jenkins-hbase4.apache.org,37011,1676087618428' in zk 2023-02-11 03:55:16,382 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testCompactionWithHFileLink' has notified controller of completion 2023-02-11 03:55:16,382 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:55:16,382 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:55:16,383 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:55:16,383 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:55:16,383 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:55:16,383 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testCompactionWithHFileLink' completed. 2023-02-11 03:55:16,383 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:55:16,383 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:55:16,383 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-11 03:55:16,383 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:55:16,384 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:55:16,384 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-11 03:55:16,384 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:55:16,384 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testCompactionWithHFileLink' member 'jenkins-hbase4.apache.org,37011,1676087618428': 2023-02-11 03:55:16,384 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37011,1676087618428' released barrier for procedure'testCompactionWithHFileLink', counting down latch. Waiting for 0 more 2023-02-11 03:55:16,384 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testCompactionWithHFileLink' execution completed 2023-02-11 03:55:16,384 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 03:55:16,384 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 03:55:16,385 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testCompactionWithHFileLink 2023-02-11 03:55:16,385 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testCompactionWithHFileLinkincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 03:55:16,387 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-11 03:55:16,387 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-11 03:55:16,387 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-11 03:55:16,387 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:55:16,387 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:55:16,387 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:55:16,387 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-11 03:55:16,387 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-11 03:55:16,388 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:55:16,388 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:55:16,388 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:55:16,388 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:55:16,388 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-11 03:55:16,388 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-11 03:55:16,388 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:55:16,389 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-11 03:55:16,389 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:55:16,389 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:55:16,389 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:55:16,389 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-11 03:55:16,390 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:55:16,391 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:55:16,391 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:55:16,391 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:55:16,391 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-02-11 03:55:16,391 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink because node does not exist (not an error) 2023-02-11 03:55:16,391 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:55:16,391 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testCompactionWithHFileLink' 2023-02-11 03:55:16,391 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 03:55:16,391 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:55:16,391 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-02-11 03:55:16,391 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:55:16,391 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:55:16,391 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-11 03:55:16,391 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-11 03:55:16,391 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-11 03:55:16,391 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testCompactionWithHFileLink'' to complete. (max 300000 ms per retry) 2023-02-11 03:55:16,391 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 03:55:16,392 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:55:16,392 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:55:16,491 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 03:55:16,492 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 03:55:16,494 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.HRegion(8801): writing data to region testCompactionWithHFileLink,,1676087711806.186d87f306418fb25d295d824c1272fe. with WAL disabled. Data may be lost in the event of a crash. 2023-02-11 03:55:16,494 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.HRegion(8801): writing data to region testCompactionWithHFileLink,1,1676087711806.ea4735f297efbbfcc4cbc0bd3d6e42e0. with WAL disabled. Data may be lost in the event of a crash. 2023-02-11 03:55:16,494 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.HRegion(8801): writing data to region testCompactionWithHFileLink,2,1676087711806.621cd59809a16fd910172a9890d26263. with WAL disabled. Data may be lost in the event of a crash. 2023-02-11 03:55:16,496 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 03:55:16,498 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] procedure.ProcedureCoordinator(143): Procedure testCompactionWithHFileLink was in running list but was completed. Accepting new attempt. 2023-02-11 03:55:16,498 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] procedure.ProcedureCoordinator(165): Submitting procedure testCompactionWithHFileLink 2023-02-11 03:55:16,498 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testCompactionWithHFileLink' 2023-02-11 03:55:16,498 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:55:16,498 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testCompactionWithHFileLink' starting 'acquire' 2023-02-11 03:55:16,498 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testCompactionWithHFileLink', kicking off acquire phase on members. 2023-02-11 03:55:16,498 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-11 03:55:16,499 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-02-11 03:55:16,499 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:55:16,500 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:55:16,500 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:55:16,500 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:55:16,500 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:55:16,500 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 03:55:16,500 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-02-11 03:55:16,500 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-11 03:55:16,500 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 03:55:16,500 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-02-11 03:55:16,500 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testCompactionWithHFileLink 2023-02-11 03:55:16,501 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testCompactionWithHFileLink is still present, removing 2023-02-11 03:55:16,501 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testCompactionWithHFileLink 2023-02-11 03:55:16,501 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testCompactionWithHFileLink' with timeout 60000ms 2023-02-11 03:55:16,501 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:55:16,501 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testCompactionWithHFileLink' starting 'acquire' stage 2023-02-11 03:55:16,501 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,2,1676087711806.621cd59809a16fd910172a9890d26263. 2023-02-11 03:55:16,501 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,2,1676087711806.621cd59809a16fd910172a9890d26263. started... 2023-02-11 03:55:16,501 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-02-11 03:55:16,501 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 03:55:16,501 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,1,1676087711806.ea4735f297efbbfcc4cbc0bd3d6e42e0. 2023-02-11 03:55:16,501 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,,1676087711806.186d87f306418fb25d295d824c1272fe. 2023-02-11 03:55:16,501 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 621cd59809a16fd910172a9890d26263 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-02-11 03:55:16,501 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,1,1676087711806.ea4735f297efbbfcc4cbc0bd3d6e42e0. started... 2023-02-11 03:55:16,501 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,,1676087711806.186d87f306418fb25d295d824c1272fe. started... 2023-02-11 03:55:16,501 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-2] regionserver.HRegion(2744): Flushing ea4735f297efbbfcc4cbc0bd3d6e42e0 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-02-11 03:55:16,501 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-1] regionserver.HRegion(2744): Flushing 186d87f306418fb25d295d824c1272fe 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-02-11 03:55:16,918 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b2023021169ada67b5fa9484db7e2de93e45014c6 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b2023021169ada67b5fa9484db7e2de93e45014c6 2023-02-11 03:55:16,920 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202302115097863fecd842c6a4c0b957f23811ff to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202302115097863fecd842c6a4c0b957f23811ff 2023-02-11 03:55:16,921 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230211e861d00e5a114d6b996d9db1875d1dc1 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230211e861d00e5a114d6b996d9db1875d1dc1 2023-02-11 03:55:17,031 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-11 03:55:17,331 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/186d87f306418fb25d295d824c1272fe/.tmp/family1/55bf25ed3b544fe9969ac6430989cefc 2023-02-11 03:55:17,331 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/ea4735f297efbbfcc4cbc0bd3d6e42e0/.tmp/family1/c504159ab71244858e1768f8eb16927b 2023-02-11 03:55:17,331 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/621cd59809a16fd910172a9890d26263/.tmp/family1/3cb4dc3b66114935967cd14111320066 2023-02-11 03:55:17,475 DEBUG [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(152): Removing adapter for the MetricRegistry: RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.mob.compactions.TestMobCompactor$CompactTwoLatestHfilesCopro 2023-02-11 03:55:17,746 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2023021188469be17b5f4d66bfeeb655b336e842 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e2023021188469be17b5f4d66bfeeb655b336e842 2023-02-11 03:55:17,746 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202302114071a4681ba54ea288ea82635207f6ae to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202302114071a4681ba54ea288ea82635207f6ae 2023-02-11 03:55:17,746 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230211cc6d231f112e474eb4759acadde745f1 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230211cc6d231f112e474eb4759acadde745f1 2023-02-11 03:55:18,153 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/186d87f306418fb25d295d824c1272fe/.tmp/family2/14bfe13c870f426d986ef125637ce8f4 2023-02-11 03:55:18,153 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/ea4735f297efbbfcc4cbc0bd3d6e42e0/.tmp/family2/a77823d8bc254eb5b1130f536fa90187 2023-02-11 03:55:18,153 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/621cd59809a16fd910172a9890d26263/.tmp/family2/87ccd53b473f47b6887f864a901e3b7c 2023-02-11 03:55:18,158 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/186d87f306418fb25d295d824c1272fe/.tmp/family1/55bf25ed3b544fe9969ac6430989cefc as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/186d87f306418fb25d295d824c1272fe/family1/55bf25ed3b544fe9969ac6430989cefc 2023-02-11 03:55:18,158 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/621cd59809a16fd910172a9890d26263/.tmp/family1/3cb4dc3b66114935967cd14111320066 as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/621cd59809a16fd910172a9890d26263/family1/3cb4dc3b66114935967cd14111320066 2023-02-11 03:55:18,158 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/ea4735f297efbbfcc4cbc0bd3d6e42e0/.tmp/family1/c504159ab71244858e1768f8eb16927b as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/ea4735f297efbbfcc4cbc0bd3d6e42e0/family1/c504159ab71244858e1768f8eb16927b 2023-02-11 03:55:18,161 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/186d87f306418fb25d295d824c1272fe/family1/55bf25ed3b544fe9969ac6430989cefc, entries=4, sequenceid=13, filesize=5.4 K 2023-02-11 03:55:18,162 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/621cd59809a16fd910172a9890d26263/family1/3cb4dc3b66114935967cd14111320066, entries=4, sequenceid=13, filesize=5.4 K 2023-02-11 03:55:18,162 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/ea4735f297efbbfcc4cbc0bd3d6e42e0/family1/c504159ab71244858e1768f8eb16927b, entries=4, sequenceid=13, filesize=5.4 K 2023-02-11 03:55:18,162 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/186d87f306418fb25d295d824c1272fe/.tmp/family2/14bfe13c870f426d986ef125637ce8f4 as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/186d87f306418fb25d295d824c1272fe/family2/14bfe13c870f426d986ef125637ce8f4 2023-02-11 03:55:18,162 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/621cd59809a16fd910172a9890d26263/.tmp/family2/87ccd53b473f47b6887f864a901e3b7c as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/621cd59809a16fd910172a9890d26263/family2/87ccd53b473f47b6887f864a901e3b7c 2023-02-11 03:55:18,162 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/ea4735f297efbbfcc4cbc0bd3d6e42e0/.tmp/family2/a77823d8bc254eb5b1130f536fa90187 as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/ea4735f297efbbfcc4cbc0bd3d6e42e0/family2/a77823d8bc254eb5b1130f536fa90187 2023-02-11 03:55:18,166 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/ea4735f297efbbfcc4cbc0bd3d6e42e0/family2/a77823d8bc254eb5b1130f536fa90187, entries=2, sequenceid=13, filesize=5.1 K 2023-02-11 03:55:18,166 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/621cd59809a16fd910172a9890d26263/family2/87ccd53b473f47b6887f864a901e3b7c, entries=2, sequenceid=13, filesize=5.1 K 2023-02-11 03:55:18,166 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/186d87f306418fb25d295d824c1272fe/family2/14bfe13c870f426d986ef125637ce8f4, entries=2, sequenceid=13, filesize=5.1 K 2023-02-11 03:55:18,167 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-2] regionserver.HRegion(2947): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for ea4735f297efbbfcc4cbc0bd3d6e42e0 in 1666ms, sequenceid=13, compaction requested=true 2023-02-11 03:55:18,167 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for 621cd59809a16fd910172a9890d26263 in 1666ms, sequenceid=13, compaction requested=true 2023-02-11 03:55:18,167 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-2] regionserver.HRegion(2445): Flush status journal for ea4735f297efbbfcc4cbc0bd3d6e42e0: 2023-02-11 03:55:18,167 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 621cd59809a16fd910172a9890d26263: 2023-02-11 03:55:18,167 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-1] regionserver.HRegion(2947): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for 186d87f306418fb25d295d824c1272fe in 1666ms, sequenceid=13, compaction requested=true 2023-02-11 03:55:18,167 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,2,1676087711806.621cd59809a16fd910172a9890d26263. 2023-02-11 03:55:18,167 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-1] regionserver.HRegion(2445): Flush status journal for 186d87f306418fb25d295d824c1272fe: 2023-02-11 03:55:18,167 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,1,1676087711806.ea4735f297efbbfcc4cbc0bd3d6e42e0. 2023-02-11 03:55:18,167 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-02-11 03:55:18,167 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,,1676087711806.186d87f306418fb25d295d824c1272fe. 2023-02-11 03:55:18,167 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-02-11 03:55:18,167 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-02-11 03:55:18,167 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-02-11 03:55:18,167 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:55:18,167 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testCompactionWithHFileLink' locally acquired 2023-02-11 03:55:18,167 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37011,1676087618428' joining acquired barrier for procedure (testCompactionWithHFileLink) in zk 2023-02-11 03:55:18,169 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-11 03:55:18,169 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:55:18,169 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:55:18,169 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:55:18,169 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:55:18,169 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-11 03:55:18,169 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testCompactionWithHFileLink' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 03:55:18,169 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:55:18,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:55:18,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-11 03:55:18,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:55:18,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:55:18,170 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37011,1676087618428' joining acquired barrier for procedure 'testCompactionWithHFileLink' on coordinator 2023-02-11 03:55:18,170 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@e8090dd[Count = 0] remaining members to acquire global barrier 2023-02-11 03:55:18,171 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testCompactionWithHFileLink' starting 'in-barrier' execution. 2023-02-11 03:55:18,171 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-11 03:55:18,172 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-11 03:55:18,173 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-11 03:55:18,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-11 03:55:18,173 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testCompactionWithHFileLink' received 'reached' from coordinator. 2023-02-11 03:55:18,173 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testCompactionWithHFileLink' locally completed 2023-02-11 03:55:18,173 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testCompactionWithHFileLink' completed for member 'jenkins-hbase4.apache.org,37011,1676087618428' in zk 2023-02-11 03:55:18,173 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:55:18,173 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 03:55:18,177 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:55:18,177 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testCompactionWithHFileLink' has notified controller of completion 2023-02-11 03:55:18,177 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:55:18,177 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:55:18,177 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:55:18,177 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:55:18,177 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testCompactionWithHFileLink' completed. 2023-02-11 03:55:18,177 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:55:18,177 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:55:18,178 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-11 03:55:18,178 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:55:18,178 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:55:18,178 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-11 03:55:18,178 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:55:18,179 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testCompactionWithHFileLink' member 'jenkins-hbase4.apache.org,37011,1676087618428': 2023-02-11 03:55:18,179 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37011,1676087618428' released barrier for procedure'testCompactionWithHFileLink', counting down latch. Waiting for 0 more 2023-02-11 03:55:18,179 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testCompactionWithHFileLink' execution completed 2023-02-11 03:55:18,179 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 03:55:18,179 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 03:55:18,179 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testCompactionWithHFileLink 2023-02-11 03:55:18,179 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testCompactionWithHFileLinkincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 03:55:18,180 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-11 03:55:18,180 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-11 03:55:18,180 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:55:18,180 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:55:18,180 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-11 03:55:18,180 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:55:18,180 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-11 03:55:18,180 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-11 03:55:18,181 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:55:18,181 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:55:18,181 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:55:18,181 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:55:18,181 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-11 03:55:18,181 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-11 03:55:18,181 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:55:18,181 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-11 03:55:18,181 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:55:18,181 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:55:18,182 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:55:18,182 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-11 03:55:18,182 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:55:18,183 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:55:18,183 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:55:18,183 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:55:18,183 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-02-11 03:55:18,183 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:55:18,183 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:55:18,183 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:55:18,183 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testCompactionWithHFileLink' 2023-02-11 03:55:18,184 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 03:55:18,183 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-02-11 03:55:18,184 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:55:18,184 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:55:18,184 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:55:18,184 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-11 03:55:18,184 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-11 03:55:18,184 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testCompactionWithHFileLink'' to complete. (max 300000 ms per retry) 2023-02-11 03:55:18,184 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 03:55:18,184 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-11 03:55:18,284 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 03:55:18,285 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 03:55:18,286 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.HRegion(8801): writing data to region testCompactionWithHFileLink,,1676087711806.186d87f306418fb25d295d824c1272fe. with WAL disabled. Data may be lost in the event of a crash. 2023-02-11 03:55:18,286 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.HRegion(8801): writing data to region testCompactionWithHFileLink,1,1676087711806.ea4735f297efbbfcc4cbc0bd3d6e42e0. with WAL disabled. Data may be lost in the event of a crash. 2023-02-11 03:55:18,286 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.HRegion(8801): writing data to region testCompactionWithHFileLink,2,1676087711806.621cd59809a16fd910172a9890d26263. with WAL disabled. Data may be lost in the event of a crash. 2023-02-11 03:55:18,289 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 03:55:18,291 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] procedure.ProcedureCoordinator(143): Procedure testCompactionWithHFileLink was in running list but was completed. Accepting new attempt. 2023-02-11 03:55:18,291 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] procedure.ProcedureCoordinator(165): Submitting procedure testCompactionWithHFileLink 2023-02-11 03:55:18,291 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testCompactionWithHFileLink' 2023-02-11 03:55:18,291 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:55:18,291 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testCompactionWithHFileLink' starting 'acquire' 2023-02-11 03:55:18,291 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testCompactionWithHFileLink', kicking off acquire phase on members. 2023-02-11 03:55:18,291 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-11 03:55:18,291 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-02-11 03:55:18,293 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:55:18,293 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:55:18,293 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:55:18,293 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:55:18,293 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:55:18,293 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 03:55:18,293 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-02-11 03:55:18,293 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-11 03:55:18,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 03:55:18,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-02-11 03:55:18,294 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testCompactionWithHFileLink 2023-02-11 03:55:18,294 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testCompactionWithHFileLink is still present, removing 2023-02-11 03:55:18,294 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testCompactionWithHFileLink 2023-02-11 03:55:18,294 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testCompactionWithHFileLink' with timeout 60000ms 2023-02-11 03:55:18,294 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:55:18,294 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testCompactionWithHFileLink' starting 'acquire' stage 2023-02-11 03:55:18,294 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,2,1676087711806.621cd59809a16fd910172a9890d26263. 2023-02-11 03:55:18,294 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,,1676087711806.186d87f306418fb25d295d824c1272fe. 2023-02-11 03:55:18,294 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-02-11 03:55:18,294 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 03:55:18,294 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,,1676087711806.186d87f306418fb25d295d824c1272fe. started... 2023-02-11 03:55:18,294 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,1,1676087711806.ea4735f297efbbfcc4cbc0bd3d6e42e0. 2023-02-11 03:55:18,295 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,1,1676087711806.ea4735f297efbbfcc4cbc0bd3d6e42e0. started... 2023-02-11 03:55:18,294 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,2,1676087711806.621cd59809a16fd910172a9890d26263. started... 2023-02-11 03:55:18,295 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-1] regionserver.HRegion(2744): Flushing 186d87f306418fb25d295d824c1272fe 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-02-11 03:55:18,295 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-2] regionserver.HRegion(2744): Flushing ea4735f297efbbfcc4cbc0bd3d6e42e0 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-02-11 03:55:18,295 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 621cd59809a16fd910172a9890d26263 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-02-11 03:55:18,708 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230211101da7200e8b4cf48af3f9e6b626325b to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230211101da7200e8b4cf48af3f9e6b626325b 2023-02-11 03:55:18,709 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c2023021148e3d369a1484149bf86dd1e2169b387 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c2023021148e3d369a1484149bf86dd1e2169b387 2023-02-11 03:55:18,709 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b2023021143a646d95a904a22a34aeef0a1d76a19 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b2023021143a646d95a904a22a34aeef0a1d76a19 2023-02-11 03:55:19,114 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/186d87f306418fb25d295d824c1272fe/.tmp/family1/1363e2a6dd1f49e0a9d550f758d2bcac 2023-02-11 03:55:19,115 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/ea4735f297efbbfcc4cbc0bd3d6e42e0/.tmp/family1/3088ce87ec264e38859de9f824a4ab99 2023-02-11 03:55:19,115 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/621cd59809a16fd910172a9890d26263/.tmp/family1/42ea1ea361834322b07cb7fa95ee893f 2023-02-11 03:55:19,532 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202302116fc53053458b4569a977face277d8136 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202302116fc53053458b4569a977face277d8136 2023-02-11 03:55:19,532 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230211e4d7df1436cd4bb0a383daae94da9f0f to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230211e4d7df1436cd4bb0a383daae94da9f0f 2023-02-11 03:55:19,532 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230211ace870e2042e406dbd52145d1caaaf37 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230211ace870e2042e406dbd52145d1caaaf37 2023-02-11 03:55:19,938 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/186d87f306418fb25d295d824c1272fe/.tmp/family2/203bb908b48a4908b252cec8e0b7c5e7 2023-02-11 03:55:19,938 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/ea4735f297efbbfcc4cbc0bd3d6e42e0/.tmp/family2/5b41b5f362864186bb9f20144e3bc4c5 2023-02-11 03:55:19,938 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/621cd59809a16fd910172a9890d26263/.tmp/family2/32530f87de8648c0ab8a83636ca1711f 2023-02-11 03:55:19,942 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/621cd59809a16fd910172a9890d26263/.tmp/family1/42ea1ea361834322b07cb7fa95ee893f as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/621cd59809a16fd910172a9890d26263/family1/42ea1ea361834322b07cb7fa95ee893f 2023-02-11 03:55:19,943 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/186d87f306418fb25d295d824c1272fe/.tmp/family1/1363e2a6dd1f49e0a9d550f758d2bcac as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/186d87f306418fb25d295d824c1272fe/family1/1363e2a6dd1f49e0a9d550f758d2bcac 2023-02-11 03:55:19,943 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/ea4735f297efbbfcc4cbc0bd3d6e42e0/.tmp/family1/3088ce87ec264e38859de9f824a4ab99 as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/ea4735f297efbbfcc4cbc0bd3d6e42e0/family1/3088ce87ec264e38859de9f824a4ab99 2023-02-11 03:55:19,947 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/ea4735f297efbbfcc4cbc0bd3d6e42e0/family1/3088ce87ec264e38859de9f824a4ab99, entries=4, sequenceid=17, filesize=5.4 K 2023-02-11 03:55:19,947 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/186d87f306418fb25d295d824c1272fe/family1/1363e2a6dd1f49e0a9d550f758d2bcac, entries=4, sequenceid=17, filesize=5.4 K 2023-02-11 03:55:19,947 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/621cd59809a16fd910172a9890d26263/family1/42ea1ea361834322b07cb7fa95ee893f, entries=4, sequenceid=17, filesize=5.4 K 2023-02-11 03:55:19,947 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/186d87f306418fb25d295d824c1272fe/.tmp/family2/203bb908b48a4908b252cec8e0b7c5e7 as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/186d87f306418fb25d295d824c1272fe/family2/203bb908b48a4908b252cec8e0b7c5e7 2023-02-11 03:55:19,948 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/621cd59809a16fd910172a9890d26263/.tmp/family2/32530f87de8648c0ab8a83636ca1711f as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/621cd59809a16fd910172a9890d26263/family2/32530f87de8648c0ab8a83636ca1711f 2023-02-11 03:55:19,948 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/ea4735f297efbbfcc4cbc0bd3d6e42e0/.tmp/family2/5b41b5f362864186bb9f20144e3bc4c5 as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/ea4735f297efbbfcc4cbc0bd3d6e42e0/family2/5b41b5f362864186bb9f20144e3bc4c5 2023-02-11 03:55:19,956 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/186d87f306418fb25d295d824c1272fe/family2/203bb908b48a4908b252cec8e0b7c5e7, entries=2, sequenceid=17, filesize=5.1 K 2023-02-11 03:55:19,957 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/621cd59809a16fd910172a9890d26263/family2/32530f87de8648c0ab8a83636ca1711f, entries=2, sequenceid=17, filesize=5.1 K 2023-02-11 03:55:19,957 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-1] regionserver.HRegion(2947): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for 186d87f306418fb25d295d824c1272fe in 1662ms, sequenceid=17, compaction requested=true 2023-02-11 03:55:19,957 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-1] regionserver.HRegion(2445): Flush status journal for 186d87f306418fb25d295d824c1272fe: 2023-02-11 03:55:19,957 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,,1676087711806.186d87f306418fb25d295d824c1272fe. 2023-02-11 03:55:19,957 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for 621cd59809a16fd910172a9890d26263 in 1662ms, sequenceid=17, compaction requested=true 2023-02-11 03:55:19,957 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-02-11 03:55:19,958 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 621cd59809a16fd910172a9890d26263: 2023-02-11 03:55:19,958 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,2,1676087711806.621cd59809a16fd910172a9890d26263. 2023-02-11 03:55:19,958 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-02-11 03:55:19,961 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/ea4735f297efbbfcc4cbc0bd3d6e42e0/family2/5b41b5f362864186bb9f20144e3bc4c5, entries=2, sequenceid=17, filesize=5.1 K 2023-02-11 03:55:19,962 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-2] regionserver.HRegion(2947): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for ea4735f297efbbfcc4cbc0bd3d6e42e0 in 1667ms, sequenceid=17, compaction requested=true 2023-02-11 03:55:19,962 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-2] regionserver.HRegion(2445): Flush status journal for ea4735f297efbbfcc4cbc0bd3d6e42e0: 2023-02-11 03:55:19,962 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,1,1676087711806.ea4735f297efbbfcc4cbc0bd3d6e42e0. 2023-02-11 03:55:19,962 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-02-11 03:55:19,962 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-02-11 03:55:19,962 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:55:19,962 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testCompactionWithHFileLink' locally acquired 2023-02-11 03:55:19,962 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37011,1676087618428' joining acquired barrier for procedure (testCompactionWithHFileLink) in zk 2023-02-11 03:55:19,965 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-11 03:55:19,965 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:55:19,965 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:55:19,965 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:55:19,965 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:55:19,965 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-11 03:55:19,965 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testCompactionWithHFileLink' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 03:55:19,965 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:55:19,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:55:19,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-11 03:55:19,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:55:19,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:55:19,966 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37011,1676087618428' joining acquired barrier for procedure 'testCompactionWithHFileLink' on coordinator 2023-02-11 03:55:19,966 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7ec2af49[Count = 0] remaining members to acquire global barrier 2023-02-11 03:55:19,966 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testCompactionWithHFileLink' starting 'in-barrier' execution. 2023-02-11 03:55:19,966 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-11 03:55:19,967 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-11 03:55:19,967 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-11 03:55:19,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-11 03:55:19,968 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testCompactionWithHFileLink' received 'reached' from coordinator. 2023-02-11 03:55:19,968 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testCompactionWithHFileLink' locally completed 2023-02-11 03:55:19,968 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testCompactionWithHFileLink' completed for member 'jenkins-hbase4.apache.org,37011,1676087618428' in zk 2023-02-11 03:55:19,968 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:55:19,968 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 03:55:19,969 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testCompactionWithHFileLink' has notified controller of completion 2023-02-11 03:55:19,969 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:55:19,969 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:55:19,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:55:19,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:55:19,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:55:19,969 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testCompactionWithHFileLink' completed. 2023-02-11 03:55:19,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:55:19,970 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:55:19,970 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-11 03:55:19,970 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:55:19,970 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:55:19,970 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-11 03:55:19,971 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:55:19,971 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testCompactionWithHFileLink' member 'jenkins-hbase4.apache.org,37011,1676087618428': 2023-02-11 03:55:19,971 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testCompactionWithHFileLink' execution completed 2023-02-11 03:55:19,971 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 03:55:19,971 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 03:55:19,971 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37011,1676087618428' released barrier for procedure'testCompactionWithHFileLink', counting down latch. Waiting for 0 more 2023-02-11 03:55:19,971 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testCompactionWithHFileLink 2023-02-11 03:55:19,971 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testCompactionWithHFileLinkincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 03:55:19,972 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-11 03:55:19,972 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-11 03:55:19,972 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-11 03:55:19,973 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:55:19,973 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:55:19,973 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-11 03:55:19,973 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-11 03:55:19,973 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:55:19,973 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:55:19,973 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:55:19,973 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:55:19,973 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:55:19,973 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-11 03:55:19,973 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-11 03:55:19,973 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:55:19,974 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-11 03:55:19,974 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:55:19,974 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:55:19,974 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:55:19,974 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-11 03:55:19,975 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:55:19,977 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:55:19,977 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:55:19,977 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:55:19,977 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-02-11 03:55:19,977 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testCompactionWithHFileLink' 2023-02-11 03:55:19,977 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink because node does not exist (not an error) 2023-02-11 03:55:19,977 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 03:55:19,977 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-02-11 03:55:19,977 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:55:19,977 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:55:19,977 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:55:19,977 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:55:19,977 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-11 03:55:19,977 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testCompactionWithHFileLink'' to complete. (max 300000 ms per retry) 2023-02-11 03:55:19,978 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 03:55:19,977 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-11 03:55:19,978 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:55:19,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:55:19,978 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-11 03:55:20,078 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 03:55:20,079 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 03:55:20,089 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] master.MasterRpcServices(1580): Client=jenkins//172.31.14.131 snapshot request for:{ ss=snaptb-1676087720079 table=testCompactionWithHFileLink type=FLUSH ttl=0 } 2023-02-11 03:55:20,089 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] snapshot.SnapshotDescriptionUtils(316): Creation time not specified, setting to:1676087720089 (current time:1676087720089). 2023-02-11 03:55:20,089 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] snapshot.SnapshotDescriptionUtils(332): Snapshot current TTL value: 0 resetting it to default value: 0 2023-02-11 03:55:20,090 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] zookeeper.ReadOnlyZKClient(139): Connect 0x74ddf32e to 127.0.0.1:53383 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-11 03:55:20,095 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@308b4961, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-11 03:55:20,099 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-11 03:55:20,102 INFO [RS-EventLoopGroup-3-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:60566, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-11 03:55:20,102 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x74ddf32e to 127.0.0.1:53383 2023-02-11 03:55:20,102 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-11 03:55:20,103 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] snapshot.SnapshotManager(601): No existing snapshot, attempting snapshot... 2023-02-11 03:55:20,104 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] snapshot.SnapshotManager(648): Table enabled, starting distributed snapshots for { ss=snaptb-1676087720079 table=testCompactionWithHFileLink type=FLUSH ttl=0 } 2023-02-11 03:55:20,112 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] procedure2.ProcedureExecutor(1029): Stored pid=86, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE 2023-02-11 03:55:20,112 DEBUG [PEWorker-2] locking.LockProcedure(309): LOCKED pid=86, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE 2023-02-11 03:55:20,113 INFO [PEWorker-2] procedure2.TimeoutExecutorThread(81): ADDED pid=86, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE; timeout=600000, timestamp=1676088320113 2023-02-11 03:55:20,113 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] snapshot.SnapshotManager(653): Started snapshot: { ss=snaptb-1676087720079 table=testCompactionWithHFileLink type=FLUSH ttl=0 } 2023-02-11 03:55:20,113 INFO [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.TakeSnapshotHandler(174): Running FLUSH table snapshot snaptb-1676087720079 C_M_SNAPSHOT_TABLE on table testCompactionWithHFileLink 2023-02-11 03:55:20,114 DEBUG [PEWorker-4] locking.LockProcedure(242): UNLOCKED pid=86, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE 2023-02-11 03:55:20,115 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] procedure2.ProcedureExecutor(1029): Stored pid=87, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=SHARED 2023-02-11 03:55:20,115 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=86, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE in 3 msec 2023-02-11 03:55:20,116 DEBUG [PEWorker-4] locking.LockProcedure(309): LOCKED pid=87, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=SHARED 2023-02-11 03:55:20,116 INFO [PEWorker-4] procedure2.TimeoutExecutorThread(81): ADDED pid=87, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=SHARED; timeout=600000, timestamp=1676088320116 2023-02-11 03:55:20,118 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2418): Waiting a max of 300000 ms for snapshot '{ ss=snaptb-1676087720079 table=testCompactionWithHFileLink type=FLUSH ttl=0 }'' to complete. (max 300000 ms per retry) 2023-02-11 03:55:20,118 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2428): (#1) Sleeping: 100ms while waiting for snapshot completion. 2023-02-11 03:55:20,218 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-02-11 03:55:20,221 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] master.MasterRpcServices(1209): Checking to see if snapshot from request:{ ss=snaptb-1676087720079 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-02-11 03:55:20,222 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] snapshot.SnapshotManager(404): Snapshoting '{ ss=snaptb-1676087720079 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' is still in progress! 2023-02-11 03:55:20,222 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2428): (#2) Sleeping: 200ms while waiting for snapshot completion. 2023-02-11 03:55:20,423 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-02-11 03:55:20,423 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] master.MasterRpcServices(1209): Checking to see if snapshot from request:{ ss=snaptb-1676087720079 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-02-11 03:55:20,424 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] snapshot.SnapshotManager(404): Snapshoting '{ ss=snaptb-1676087720079 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' is still in progress! 2023-02-11 03:55:20,424 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2428): (#3) Sleeping: 300ms while waiting for snapshot completion. 2023-02-11 03:55:20,523 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] procedure.ProcedureCoordinator(165): Submitting procedure snaptb-1676087720079 2023-02-11 03:55:20,523 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'snaptb-1676087720079' 2023-02-11 03:55:20,523 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 300000 ms 2023-02-11 03:55:20,523 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'snaptb-1676087720079' starting 'acquire' 2023-02-11 03:55:20,523 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'snaptb-1676087720079', kicking off acquire phase on members. 2023-02-11 03:55:20,524 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/online-snapshot/abort/snaptb-1676087720079 2023-02-11 03:55:20,524 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/online-snapshot/acquired/snaptb-1676087720079 2023-02-11 03:55:20,525 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/online-snapshot/acquired/snaptb-1676087720079/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:55:20,525 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/online-snapshot/acquired 2023-02-11 03:55:20,525 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/online-snapshot/acquired 2023-02-11 03:55:20,525 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-02-11 03:55:20,525 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/online-snapshot/acquired/snaptb-1676087720079/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:55:20,525 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 03:55:20,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/online-snapshot/acquired/snaptb-1676087720079 2023-02-11 03:55:20,526 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/online-snapshot/abort/snaptb-1676087720079 2023-02-11 03:55:20,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 68 2023-02-11 03:55:20,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/online-snapshot/acquired/snaptb-1676087720079 2023-02-11 03:55:20,526 DEBUG [zk-event-processor-pool-0] snapshot.RegionServerSnapshotManager(175): Launching subprocedure for snapshot snaptb-1676087720079 from table testCompactionWithHFileLink type FLUSH 2023-02-11 03:55:20,529 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:snaptb-1676087720079 2023-02-11 03:55:20,529 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'snaptb-1676087720079' with timeout 300000ms 2023-02-11 03:55:20,529 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 300000 ms 2023-02-11 03:55:20,529 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'snaptb-1676087720079' starting 'acquire' stage 2023-02-11 03:55:20,529 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'snaptb-1676087720079' locally acquired 2023-02-11 03:55:20,529 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37011,1676087618428' joining acquired barrier for procedure (snaptb-1676087720079) in zk 2023-02-11 03:55:20,532 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/online-snapshot/reached/snaptb-1676087720079 2023-02-11 03:55:20,532 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/online-snapshot/acquired/snaptb-1676087720079/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:55:20,532 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/online-snapshot/acquired/snaptb-1676087720079/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:55:20,532 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:55:20,532 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/online-snapshot 2023-02-11 03:55:20,532 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/online-snapshot/reached/snaptb-1676087720079 2023-02-11 03:55:20,532 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'snaptb-1676087720079' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 03:55:20,532 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:55:20,532 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:55:20,533 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----snaptb-1676087720079 2023-02-11 03:55:20,533 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:55:20,533 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:55:20,533 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37011,1676087618428' joining acquired barrier for procedure 'snaptb-1676087720079' on coordinator 2023-02-11 03:55:20,533 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3ec6a5fc[Count = 0] remaining members to acquire global barrier 2023-02-11 03:55:20,533 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'snaptb-1676087720079' starting 'in-barrier' execution. 2023-02-11 03:55:20,533 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/online-snapshot/reached/snaptb-1676087720079 2023-02-11 03:55:20,534 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/online-snapshot/reached/snaptb-1676087720079 2023-02-11 03:55:20,534 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/online-snapshot/reached/snaptb-1676087720079 2023-02-11 03:55:20,534 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/online-snapshot/reached/snaptb-1676087720079 2023-02-11 03:55:20,534 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/online-snapshot/reached/snaptb-1676087720079/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:55:20,535 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 03:55:20,534 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'snaptb-1676087720079' received 'reached' from coordinator. 2023-02-11 03:55:20,535 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-snapshot-pool-0] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(97): Starting snapshot operation on testCompactionWithHFileLink,2,1676087711806.621cd59809a16fd910172a9890d26263. 2023-02-11 03:55:20,535 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-snapshot-pool-1] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(97): Starting snapshot operation on testCompactionWithHFileLink,,1676087711806.186d87f306418fb25d295d824c1272fe. 2023-02-11 03:55:20,536 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] snapshot.FlushSnapshotSubprocedure(170): Flush Snapshot Tasks submitted for 3 regions 2023-02-11 03:55:20,536 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-snapshot-pool-2] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(97): Starting snapshot operation on testCompactionWithHFileLink,1,1676087711806.ea4735f297efbbfcc4cbc0bd3d6e42e0. 2023-02-11 03:55:20,536 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] snapshot.RegionServerSnapshotManager$SnapshotSubprocedurePool(301): Waiting for local region snapshots to finish. 2023-02-11 03:55:20,536 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-snapshot-pool-2] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(110): Flush Snapshotting region testCompactionWithHFileLink,1,1676087711806.ea4735f297efbbfcc4cbc0bd3d6e42e0. started... 2023-02-11 03:55:20,536 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-snapshot-pool-1] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(110): Flush Snapshotting region testCompactionWithHFileLink,,1676087711806.186d87f306418fb25d295d824c1272fe. started... 2023-02-11 03:55:20,536 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-snapshot-pool-0] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(110): Flush Snapshotting region testCompactionWithHFileLink,2,1676087711806.621cd59809a16fd910172a9890d26263. started... 2023-02-11 03:55:20,536 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-snapshot-pool-1] regionserver.HRegion(2445): Flush status journal for 186d87f306418fb25d295d824c1272fe: 2023-02-11 03:55:20,536 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-snapshot-pool-2] regionserver.HRegion(2445): Flush status journal for ea4735f297efbbfcc4cbc0bd3d6e42e0: 2023-02-11 03:55:20,536 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-snapshot-pool-0] regionserver.HRegion(2445): Flush status journal for 621cd59809a16fd910172a9890d26263: 2023-02-11 03:55:20,537 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-snapshot-pool-2] snapshot.SnapshotManifest(238): Storing 'testCompactionWithHFileLink,1,1676087711806.ea4735f297efbbfcc4cbc0bd3d6e42e0.' region-info for snapshot=snaptb-1676087720079 2023-02-11 03:55:20,537 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-snapshot-pool-1] snapshot.SnapshotManifest(238): Storing 'testCompactionWithHFileLink,,1676087711806.186d87f306418fb25d295d824c1272fe.' region-info for snapshot=snaptb-1676087720079 2023-02-11 03:55:20,537 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-snapshot-pool-0] snapshot.SnapshotManifest(238): Storing 'testCompactionWithHFileLink,2,1676087711806.621cd59809a16fd910172a9890d26263.' region-info for snapshot=snaptb-1676087720079 2023-02-11 03:55:20,542 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-snapshot-pool-1] snapshot.SnapshotManifest(243): Creating references for hfiles 2023-02-11 03:55:20,542 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-snapshot-pool-0] snapshot.SnapshotManifest(243): Creating references for hfiles 2023-02-11 03:55:20,542 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-snapshot-pool-2] snapshot.SnapshotManifest(243): Creating references for hfiles 2023-02-11 03:55:20,545 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-snapshot-pool-1] snapshot.SnapshotManifest(253): Adding snapshot references for [hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/186d87f306418fb25d295d824c1272fe/family1/2e6dcfa9c78543fcb2273f9d67b88e36, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/186d87f306418fb25d295d824c1272fe/family1/4e9479861f4540a6b8a67dee5f4b8c2c, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/186d87f306418fb25d295d824c1272fe/family1/55bf25ed3b544fe9969ac6430989cefc, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/186d87f306418fb25d295d824c1272fe/family1/1363e2a6dd1f49e0a9d550f758d2bcac] hfiles 2023-02-11 03:55:20,545 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-snapshot-pool-2] snapshot.SnapshotManifest(253): Adding snapshot references for [hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/ea4735f297efbbfcc4cbc0bd3d6e42e0/family1/92743cf8525b4b5d877b358e975a3bec, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/ea4735f297efbbfcc4cbc0bd3d6e42e0/family1/0a642f51d6d94a79b4c5663961dee5c0, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/ea4735f297efbbfcc4cbc0bd3d6e42e0/family1/c504159ab71244858e1768f8eb16927b, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/ea4735f297efbbfcc4cbc0bd3d6e42e0/family1/3088ce87ec264e38859de9f824a4ab99] hfiles 2023-02-11 03:55:20,545 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-snapshot-pool-1] snapshot.SnapshotManifest(262): Adding reference for file (1/4): hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/186d87f306418fb25d295d824c1272fe/family1/2e6dcfa9c78543fcb2273f9d67b88e36 for snapshot=snaptb-1676087720079 2023-02-11 03:55:20,545 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-snapshot-pool-2] snapshot.SnapshotManifest(262): Adding reference for file (1/4): hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/ea4735f297efbbfcc4cbc0bd3d6e42e0/family1/92743cf8525b4b5d877b358e975a3bec for snapshot=snaptb-1676087720079 2023-02-11 03:55:20,545 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-snapshot-pool-0] snapshot.SnapshotManifest(253): Adding snapshot references for [hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/621cd59809a16fd910172a9890d26263/family1/a54c49bae3b34318920fc4197db7ae61, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/621cd59809a16fd910172a9890d26263/family1/051a49ec8101444f8e462520d66fa021, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/621cd59809a16fd910172a9890d26263/family1/3cb4dc3b66114935967cd14111320066, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/621cd59809a16fd910172a9890d26263/family1/42ea1ea361834322b07cb7fa95ee893f] hfiles 2023-02-11 03:55:20,546 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-snapshot-pool-0] snapshot.SnapshotManifest(262): Adding reference for file (1/4): hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/621cd59809a16fd910172a9890d26263/family1/a54c49bae3b34318920fc4197db7ae61 for snapshot=snaptb-1676087720079 2023-02-11 03:55:20,549 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-snapshot-pool-0] snapshot.SnapshotManifest(262): Adding reference for file (2/4): hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/621cd59809a16fd910172a9890d26263/family1/051a49ec8101444f8e462520d66fa021 for snapshot=snaptb-1676087720079 2023-02-11 03:55:20,549 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-snapshot-pool-1] snapshot.SnapshotManifest(262): Adding reference for file (2/4): hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/186d87f306418fb25d295d824c1272fe/family1/4e9479861f4540a6b8a67dee5f4b8c2c for snapshot=snaptb-1676087720079 2023-02-11 03:55:20,549 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-snapshot-pool-2] snapshot.SnapshotManifest(262): Adding reference for file (2/4): hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/ea4735f297efbbfcc4cbc0bd3d6e42e0/family1/0a642f51d6d94a79b4c5663961dee5c0 for snapshot=snaptb-1676087720079 2023-02-11 03:55:20,549 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-snapshot-pool-1] snapshot.SnapshotManifest(262): Adding reference for file (3/4): hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/186d87f306418fb25d295d824c1272fe/family1/55bf25ed3b544fe9969ac6430989cefc for snapshot=snaptb-1676087720079 2023-02-11 03:55:20,549 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-snapshot-pool-2] snapshot.SnapshotManifest(262): Adding reference for file (3/4): hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/ea4735f297efbbfcc4cbc0bd3d6e42e0/family1/c504159ab71244858e1768f8eb16927b for snapshot=snaptb-1676087720079 2023-02-11 03:55:20,549 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-snapshot-pool-0] snapshot.SnapshotManifest(262): Adding reference for file (3/4): hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/621cd59809a16fd910172a9890d26263/family1/3cb4dc3b66114935967cd14111320066 for snapshot=snaptb-1676087720079 2023-02-11 03:55:20,549 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-snapshot-pool-2] snapshot.SnapshotManifest(262): Adding reference for file (4/4): hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/ea4735f297efbbfcc4cbc0bd3d6e42e0/family1/3088ce87ec264e38859de9f824a4ab99 for snapshot=snaptb-1676087720079 2023-02-11 03:55:20,549 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-snapshot-pool-0] snapshot.SnapshotManifest(262): Adding reference for file (4/4): hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/621cd59809a16fd910172a9890d26263/family1/42ea1ea361834322b07cb7fa95ee893f for snapshot=snaptb-1676087720079 2023-02-11 03:55:20,549 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-snapshot-pool-1] snapshot.SnapshotManifest(262): Adding reference for file (4/4): hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/186d87f306418fb25d295d824c1272fe/family1/1363e2a6dd1f49e0a9d550f758d2bcac for snapshot=snaptb-1676087720079 2023-02-11 03:55:20,549 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-snapshot-pool-0] snapshot.SnapshotManifest(253): Adding snapshot references for [hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/621cd59809a16fd910172a9890d26263/family2/774f5f8b4d2e463f983d817bdedf591e, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/621cd59809a16fd910172a9890d26263/family2/e000a979eed34f31ba1476e0d7c949da, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/621cd59809a16fd910172a9890d26263/family2/87ccd53b473f47b6887f864a901e3b7c, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/621cd59809a16fd910172a9890d26263/family2/32530f87de8648c0ab8a83636ca1711f] hfiles 2023-02-11 03:55:20,549 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-snapshot-pool-1] snapshot.SnapshotManifest(253): Adding snapshot references for [hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/186d87f306418fb25d295d824c1272fe/family2/48173b765b124663a97b2f43b90e8a88, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/186d87f306418fb25d295d824c1272fe/family2/039e3e0c12c742f29c5ea86a66e98c44, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/186d87f306418fb25d295d824c1272fe/family2/14bfe13c870f426d986ef125637ce8f4, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/186d87f306418fb25d295d824c1272fe/family2/203bb908b48a4908b252cec8e0b7c5e7] hfiles 2023-02-11 03:55:20,549 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-snapshot-pool-1] snapshot.SnapshotManifest(262): Adding reference for file (1/4): hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/186d87f306418fb25d295d824c1272fe/family2/48173b765b124663a97b2f43b90e8a88 for snapshot=snaptb-1676087720079 2023-02-11 03:55:20,549 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-snapshot-pool-0] snapshot.SnapshotManifest(262): Adding reference for file (1/4): hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/621cd59809a16fd910172a9890d26263/family2/774f5f8b4d2e463f983d817bdedf591e for snapshot=snaptb-1676087720079 2023-02-11 03:55:20,549 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-snapshot-pool-2] snapshot.SnapshotManifest(253): Adding snapshot references for [hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/ea4735f297efbbfcc4cbc0bd3d6e42e0/family2/7141c179434b4a0f816573a88da1caf7, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/ea4735f297efbbfcc4cbc0bd3d6e42e0/family2/045314dc19574fe9979fbedc8e14acf8, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/ea4735f297efbbfcc4cbc0bd3d6e42e0/family2/a77823d8bc254eb5b1130f536fa90187, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/ea4735f297efbbfcc4cbc0bd3d6e42e0/family2/5b41b5f362864186bb9f20144e3bc4c5] hfiles 2023-02-11 03:55:20,549 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-snapshot-pool-2] snapshot.SnapshotManifest(262): Adding reference for file (1/4): hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/ea4735f297efbbfcc4cbc0bd3d6e42e0/family2/7141c179434b4a0f816573a88da1caf7 for snapshot=snaptb-1676087720079 2023-02-11 03:55:20,549 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-snapshot-pool-0] snapshot.SnapshotManifest(262): Adding reference for file (2/4): hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/621cd59809a16fd910172a9890d26263/family2/e000a979eed34f31ba1476e0d7c949da for snapshot=snaptb-1676087720079 2023-02-11 03:55:20,549 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-snapshot-pool-1] snapshot.SnapshotManifest(262): Adding reference for file (2/4): hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/186d87f306418fb25d295d824c1272fe/family2/039e3e0c12c742f29c5ea86a66e98c44 for snapshot=snaptb-1676087720079 2023-02-11 03:55:20,550 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-snapshot-pool-0] snapshot.SnapshotManifest(262): Adding reference for file (3/4): hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/621cd59809a16fd910172a9890d26263/family2/87ccd53b473f47b6887f864a901e3b7c for snapshot=snaptb-1676087720079 2023-02-11 03:55:20,550 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-snapshot-pool-1] snapshot.SnapshotManifest(262): Adding reference for file (3/4): hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/186d87f306418fb25d295d824c1272fe/family2/14bfe13c870f426d986ef125637ce8f4 for snapshot=snaptb-1676087720079 2023-02-11 03:55:20,550 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-snapshot-pool-2] snapshot.SnapshotManifest(262): Adding reference for file (2/4): hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/ea4735f297efbbfcc4cbc0bd3d6e42e0/family2/045314dc19574fe9979fbedc8e14acf8 for snapshot=snaptb-1676087720079 2023-02-11 03:55:20,550 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-snapshot-pool-1] snapshot.SnapshotManifest(262): Adding reference for file (4/4): hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/186d87f306418fb25d295d824c1272fe/family2/203bb908b48a4908b252cec8e0b7c5e7 for snapshot=snaptb-1676087720079 2023-02-11 03:55:20,550 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-snapshot-pool-0] snapshot.SnapshotManifest(262): Adding reference for file (4/4): hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/621cd59809a16fd910172a9890d26263/family2/32530f87de8648c0ab8a83636ca1711f for snapshot=snaptb-1676087720079 2023-02-11 03:55:20,550 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-snapshot-pool-2] snapshot.SnapshotManifest(262): Adding reference for file (3/4): hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/ea4735f297efbbfcc4cbc0bd3d6e42e0/family2/a77823d8bc254eb5b1130f536fa90187 for snapshot=snaptb-1676087720079 2023-02-11 03:55:20,550 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-snapshot-pool-2] snapshot.SnapshotManifest(262): Adding reference for file (4/4): hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/ea4735f297efbbfcc4cbc0bd3d6e42e0/family2/5b41b5f362864186bb9f20144e3bc4c5 for snapshot=snaptb-1676087720079 2023-02-11 03:55:20,724 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-02-11 03:55:20,725 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] master.MasterRpcServices(1209): Checking to see if snapshot from request:{ ss=snaptb-1676087720079 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-02-11 03:55:20,725 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] snapshot.SnapshotManager(404): Snapshoting '{ ss=snaptb-1676087720079 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' is still in progress! 2023-02-11 03:55:20,725 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2428): (#4) Sleeping: 500ms while waiting for snapshot completion. 2023-02-11 03:55:20,956 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-snapshot-pool-1] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(137): ... Flush Snapshotting region testCompactionWithHFileLink,,1676087711806.186d87f306418fb25d295d824c1272fe. completed. 2023-02-11 03:55:20,956 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-snapshot-pool-1] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(140): Closing snapshot operation on testCompactionWithHFileLink,,1676087711806.186d87f306418fb25d295d824c1272fe. 2023-02-11 03:55:20,956 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] snapshot.RegionServerSnapshotManager$SnapshotSubprocedurePool(312): Completed 1/3 local region snapshots. 2023-02-11 03:55:20,958 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-snapshot-pool-0] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(137): ... Flush Snapshotting region testCompactionWithHFileLink,2,1676087711806.621cd59809a16fd910172a9890d26263. completed. 2023-02-11 03:55:20,958 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-snapshot-pool-0] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(140): Closing snapshot operation on testCompactionWithHFileLink,2,1676087711806.621cd59809a16fd910172a9890d26263. 2023-02-11 03:55:20,958 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] snapshot.RegionServerSnapshotManager$SnapshotSubprocedurePool(312): Completed 2/3 local region snapshots. 2023-02-11 03:55:20,961 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-snapshot-pool-2] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(137): ... Flush Snapshotting region testCompactionWithHFileLink,1,1676087711806.ea4735f297efbbfcc4cbc0bd3d6e42e0. completed. 2023-02-11 03:55:20,961 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-snapshot-pool-2] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(140): Closing snapshot operation on testCompactionWithHFileLink,1,1676087711806.ea4735f297efbbfcc4cbc0bd3d6e42e0. 2023-02-11 03:55:20,961 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] snapshot.RegionServerSnapshotManager$SnapshotSubprocedurePool(312): Completed 3/3 local region snapshots. 2023-02-11 03:55:20,961 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] snapshot.RegionServerSnapshotManager$SnapshotSubprocedurePool(314): Completed 3 local region snapshots. 2023-02-11 03:55:20,961 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] snapshot.RegionServerSnapshotManager$SnapshotSubprocedurePool(345): cancelling 0 tasks for snapshot jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:55:20,962 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'snaptb-1676087720079' locally completed 2023-02-11 03:55:20,962 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'snaptb-1676087720079' completed for member 'jenkins-hbase4.apache.org,37011,1676087618428' in zk 2023-02-11 03:55:20,963 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'snaptb-1676087720079' has notified controller of completion 2023-02-11 03:55:20,963 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:55:20,963 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/online-snapshot/reached/snaptb-1676087720079/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:55:20,963 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'snaptb-1676087720079' completed. 2023-02-11 03:55:20,963 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/online-snapshot/reached/snaptb-1676087720079/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:55:20,963 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:55:20,963 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/online-snapshot 2023-02-11 03:55:20,964 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:55:20,964 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:55:20,964 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----snaptb-1676087720079 2023-02-11 03:55:20,964 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:55:20,964 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:55:20,965 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----snaptb-1676087720079 2023-02-11 03:55:20,965 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:55:20,965 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'snaptb-1676087720079' member 'jenkins-hbase4.apache.org,37011,1676087618428': 2023-02-11 03:55:20,965 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37011,1676087618428' released barrier for procedure'snaptb-1676087720079', counting down latch. Waiting for 0 more 2023-02-11 03:55:20,965 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'snaptb-1676087720079' execution completed 2023-02-11 03:55:20,965 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 03:55:20,965 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 03:55:20,965 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:snaptb-1676087720079 2023-02-11 03:55:20,965 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure snaptb-1676087720079including nodes /hbase/online-snapshot/acquired /hbase/online-snapshot/reached /hbase/online-snapshot/abort 2023-02-11 03:55:20,967 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/online-snapshot/abort/snaptb-1676087720079 2023-02-11 03:55:20,967 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/online-snapshot/abort/snaptb-1676087720079 2023-02-11 03:55:20,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/online-snapshot/abort/snaptb-1676087720079 2023-02-11 03:55:20,967 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/online-snapshot/abort/snaptb-1676087720079 2023-02-11 03:55:20,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:55:20,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/online-snapshot 2023-02-11 03:55:20,967 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/online-snapshot/abort 2023-02-11 03:55:20,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/online-snapshot/abort/snaptb-1676087720079 2023-02-11 03:55:20,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:55:20,967 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/online-snapshot/abort 2023-02-11 03:55:20,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-02-11 03:55:20,967 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on existing znode=/hbase/online-snapshot/acquired/snaptb-1676087720079/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:55:20,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----snaptb-1676087720079 2023-02-11 03:55:20,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/online-snapshot/abort/snaptb-1676087720079 2023-02-11 03:55:20,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:55:20,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----snaptb-1676087720079 2023-02-11 03:55:20,968 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on existing znode=/hbase/online-snapshot/reached/snaptb-1676087720079/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:55:20,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:55:20,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:55:20,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----snaptb-1676087720079 2023-02-11 03:55:20,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:55:20,972 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/online-snapshot/acquired/snaptb-1676087720079/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:55:20,972 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/online-snapshot/acquired 2023-02-11 03:55:20,972 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/online-snapshot/acquired/snaptb-1676087720079 2023-02-11 03:55:20,972 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/online-snapshot/acquired 2023-02-11 03:55:20,972 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:55:20,972 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/online-snapshot/abort 2023-02-11 03:55:20,972 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-02-11 03:55:20,972 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Unable to get data of znode /hbase/online-snapshot/abort/snaptb-1676087720079 because node does not exist (not an error) 2023-02-11 03:55:20,972 INFO [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.EnabledTableSnapshotHandler(97): Done waiting - online snapshot for snaptb-1676087720079 2023-02-11 03:55:20,972 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/online-snapshot/acquired/snaptb-1676087720079 2023-02-11 03:55:20,972 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/online-snapshot/reached/snaptb-1676087720079/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:55:20,972 INFO [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.EnabledTableSnapshotHandler(113): Taking snapshot for mob files in table testCompactionWithHFileLink 2023-02-11 03:55:20,972 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/online-snapshot/reached/snaptb-1676087720079 2023-02-11 03:55:20,972 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/online-snapshot/reached/snaptb-1676087720079 2023-02-11 03:55:20,972 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/online-snapshot/abort 2023-02-11 03:55:20,972 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(191): Storing mob region '{ENCODED => 0e7b6fa9e81bd62588edd7d77d6165e4, NAME => 'testCompactionWithHFileLink,.mob,0.0e7b6fa9e81bd62588edd7d77d6165e4.', STARTKEY => '.mob', ENDKEY => ''}' region-info for snapshot=snaptb-1676087720079 2023-02-11 03:55:20,972 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/online-snapshot/abort/snaptb-1676087720079 2023-02-11 03:55:20,972 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(196): Creating references for mob files 2023-02-11 03:55:20,972 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-02-11 03:55:20,973 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b2023021101c3e7c91909459eaa090b3ee73d4c03, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b2023021143a646d95a904a22a34aeef0a1d76a19, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b2023021169ada67b5fa9484db7e2de93e45014c6, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230211bbcb00e25dbb444c96d85d72f4a0a09a, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230211464ca8637b744ea09bffcf8850a4d558, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c2023021148e3d369a1484149bf86dd1e2169b387, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202302115097863fecd842c6a4c0b957f23811ff, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202302116f5da94e2cf44f26952495b89148e481, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230211101da7200e8b4cf48af3f9e6b626325b, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230211e4a3d035f9844b869e82ec0923db93f1, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230211e861d00e5a114d6b996d9db1875d1dc1, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230211efe856603053418f95a0d3b45cd7d38d] mob files 2023-02-11 03:55:20,973 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (1/12): hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b2023021101c3e7c91909459eaa090b3ee73d4c03 2023-02-11 03:55:20,973 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (2/12): hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b2023021143a646d95a904a22a34aeef0a1d76a19 2023-02-11 03:55:20,973 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (3/12): hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b2023021169ada67b5fa9484db7e2de93e45014c6 2023-02-11 03:55:20,973 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (4/12): hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230211bbcb00e25dbb444c96d85d72f4a0a09a 2023-02-11 03:55:20,973 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (5/12): hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230211464ca8637b744ea09bffcf8850a4d558 2023-02-11 03:55:20,974 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (6/12): hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c2023021148e3d369a1484149bf86dd1e2169b387 2023-02-11 03:55:20,974 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (7/12): hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202302115097863fecd842c6a4c0b957f23811ff 2023-02-11 03:55:20,974 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (8/12): hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202302116f5da94e2cf44f26952495b89148e481 2023-02-11 03:55:20,974 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (9/12): hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230211101da7200e8b4cf48af3f9e6b626325b 2023-02-11 03:55:20,974 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (10/12): hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230211e4a3d035f9844b869e82ec0923db93f1 2023-02-11 03:55:20,974 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (11/12): hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230211e861d00e5a114d6b996d9db1875d1dc1 2023-02-11 03:55:20,974 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (12/12): hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230211efe856603053418f95a0d3b45cd7d38d 2023-02-11 03:55:20,974 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230211211c0620d3924f14baf3c085690d9d43, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202302114071a4681ba54ea288ea82635207f6ae, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b2023021165313005ab884230948c8003bec6d4ef, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202302116fc53053458b4569a977face277d8136, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c202302110d01c3b1b7ff473d89c5b36246154d46, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c202302117e5060c420684526926d62b7b980770c, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230211cc6d231f112e474eb4759acadde745f1, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230211e4d7df1436cd4bb0a383daae94da9f0f, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e202302110f3eef6bbc07470d998a60ead6aad577, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e2023021188469be17b5f4d66bfeeb655b336e842, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230211ace870e2042e406dbd52145d1caaaf37, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230211cb55ca343ae8440a83b4999e2a1b2131] mob files 2023-02-11 03:55:20,975 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (1/12): hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230211211c0620d3924f14baf3c085690d9d43 2023-02-11 03:55:20,975 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (2/12): hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202302114071a4681ba54ea288ea82635207f6ae 2023-02-11 03:55:20,975 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (3/12): hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b2023021165313005ab884230948c8003bec6d4ef 2023-02-11 03:55:20,975 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (4/12): hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202302116fc53053458b4569a977face277d8136 2023-02-11 03:55:20,975 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (5/12): hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c202302110d01c3b1b7ff473d89c5b36246154d46 2023-02-11 03:55:20,975 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (6/12): hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c202302117e5060c420684526926d62b7b980770c 2023-02-11 03:55:20,975 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (7/12): hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230211cc6d231f112e474eb4759acadde745f1 2023-02-11 03:55:20,975 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (8/12): hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230211e4d7df1436cd4bb0a383daae94da9f0f 2023-02-11 03:55:20,975 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (9/12): hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e202302110f3eef6bbc07470d998a60ead6aad577 2023-02-11 03:55:20,975 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (10/12): hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e2023021188469be17b5f4d66bfeeb655b336e842 2023-02-11 03:55:20,975 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (11/12): hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230211ace870e2042e406dbd52145d1caaaf37 2023-02-11 03:55:20,975 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (12/12): hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230211cb55ca343ae8440a83b4999e2a1b2131 2023-02-11 03:55:21,225 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-02-11 03:55:21,226 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] master.MasterRpcServices(1209): Checking to see if snapshot from request:{ ss=snaptb-1676087720079 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-02-11 03:55:21,226 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] snapshot.SnapshotManager(404): Snapshoting '{ ss=snaptb-1676087720079 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' is still in progress! 2023-02-11 03:55:21,226 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2428): (#5) Sleeping: 1000ms while waiting for snapshot completion. 2023-02-11 03:55:21,380 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(484): Convert to Single Snapshot Manifest for snaptb-1676087720079 2023-02-11 03:55:21,382 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifestV1(126): No regions under directory:hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/.hbase-snapshot/.tmp/snaptb-1676087720079 2023-02-11 03:55:21,827 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotDescriptionUtils(404): Sentinel is done, just moving the snapshot from hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/.hbase-snapshot/.tmp/snaptb-1676087720079 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/.hbase-snapshot/snaptb-1676087720079 2023-02-11 03:55:22,226 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-02-11 03:55:22,227 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] master.MasterRpcServices(1209): Checking to see if snapshot from request:{ ss=snaptb-1676087720079 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-02-11 03:55:22,227 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] snapshot.SnapshotManager(404): Snapshoting '{ ss=snaptb-1676087720079 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' is still in progress! 2023-02-11 03:55:22,228 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2428): (#6) Sleeping: 2000ms while waiting for snapshot completion. 2023-02-11 03:55:22,642 INFO [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.TakeSnapshotHandler(229): Snapshot snaptb-1676087720079 of table testCompactionWithHFileLink completed 2023-02-11 03:55:22,642 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.TakeSnapshotHandler(246): Launching cleanup of working dir:hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/.hbase-snapshot/.tmp/snaptb-1676087720079 2023-02-11 03:55:22,642 ERROR [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.TakeSnapshotHandler(251): Couldn't delete snapshot working directory:hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/.hbase-snapshot/.tmp/snaptb-1676087720079 2023-02-11 03:55:22,642 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.TakeSnapshotHandler(257): Table snapshot journal : Running FLUSH table snapshot snaptb-1676087720079 C_M_SNAPSHOT_TABLE on table testCompactionWithHFileLink at 1676087720113Completed referencing HFiles for the mob region of table: testCompactionWithHFileLink at 1676087721380 (+1267 ms)Consolidate snapshot: snaptb-1676087720079 at 1676087721380Loading Region manifests for snaptb-1676087720079 at 1676087721380Writing data manifest for snaptb-1676087720079 at 1676087721391 (+11 ms)Verifying snapshot: snaptb-1676087720079 at 1676087721798 (+407 ms)Snapshot snaptb-1676087720079 of table testCompactionWithHFileLink completed at 1676087722642 (+844 ms) 2023-02-11 03:55:22,643 DEBUG [PEWorker-5] locking.LockProcedure(242): UNLOCKED pid=87, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=SHARED 2023-02-11 03:55:22,645 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=87, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=SHARED in 2.5280 sec 2023-02-11 03:55:22,977 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-11 03:55:24,228 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-02-11 03:55:24,229 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] master.MasterRpcServices(1209): Checking to see if snapshot from request:{ ss=snaptb-1676087720079 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-02-11 03:55:24,229 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] snapshot.SnapshotManager(401): Snapshot '{ ss=snaptb-1676087720079 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' has completed, notifying client. 2023-02-11 03:55:24,257 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 03:55:24,258 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] procedure.ProcedureCoordinator(143): Procedure testCompactionWithHFileLink was in running list but was completed. Accepting new attempt. 2023-02-11 03:55:24,259 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] procedure.ProcedureCoordinator(165): Submitting procedure testCompactionWithHFileLink 2023-02-11 03:55:24,259 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testCompactionWithHFileLink' 2023-02-11 03:55:24,259 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:55:24,259 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testCompactionWithHFileLink' starting 'acquire' 2023-02-11 03:55:24,259 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testCompactionWithHFileLink', kicking off acquire phase on members. 2023-02-11 03:55:24,259 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-11 03:55:24,259 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-02-11 03:55:24,260 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:55:24,260 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:55:24,260 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:55:24,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:55:24,260 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:55:24,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-02-11 03:55:24,260 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 03:55:24,261 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-11 03:55:24,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 03:55:24,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-02-11 03:55:24,261 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testCompactionWithHFileLink 2023-02-11 03:55:24,261 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testCompactionWithHFileLink is still present, removing 2023-02-11 03:55:24,261 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testCompactionWithHFileLink 2023-02-11 03:55:24,261 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testCompactionWithHFileLink' with timeout 60000ms 2023-02-11 03:55:24,261 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:55:24,261 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testCompactionWithHFileLink' starting 'acquire' stage 2023-02-11 03:55:24,262 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,2,1676087711806.621cd59809a16fd910172a9890d26263. 2023-02-11 03:55:24,262 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,,1676087711806.186d87f306418fb25d295d824c1272fe. 2023-02-11 03:55:24,262 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,1,1676087711806.ea4735f297efbbfcc4cbc0bd3d6e42e0. 2023-02-11 03:55:24,262 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-02-11 03:55:24,262 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 03:55:24,262 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,1,1676087711806.ea4735f297efbbfcc4cbc0bd3d6e42e0. started... 2023-02-11 03:55:24,262 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,,1676087711806.186d87f306418fb25d295d824c1272fe. started... 2023-02-11 03:55:24,262 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,2,1676087711806.621cd59809a16fd910172a9890d26263. started... 2023-02-11 03:55:24,262 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-1] regionserver.HRegion(2744): Flushing 186d87f306418fb25d295d824c1272fe 2/2 column families, dataSize=138 B heapSize=944 B 2023-02-11 03:55:24,262 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-2] regionserver.HRegion(2744): Flushing ea4735f297efbbfcc4cbc0bd3d6e42e0 2/2 column families, dataSize=138 B heapSize=944 B 2023-02-11 03:55:24,262 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 621cd59809a16fd910172a9890d26263 2/2 column families, dataSize=138 B heapSize=944 B 2023-02-11 03:55:25,082 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=24, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/621cd59809a16fd910172a9890d26263/.tmp/family1/fa6ce126bb87484288527c3ce84deb9c 2023-02-11 03:55:25,082 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=24, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/ea4735f297efbbfcc4cbc0bd3d6e42e0/.tmp/family1/48f103d11dc0460c87b57771c52929c1 2023-02-11 03:55:25,082 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=24, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/186d87f306418fb25d295d824c1272fe/.tmp/family1/851439bb236e4da28f252769b0d65177 2023-02-11 03:55:25,086 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 48f103d11dc0460c87b57771c52929c1 2023-02-11 03:55:25,086 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for fa6ce126bb87484288527c3ce84deb9c 2023-02-11 03:55:25,091 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 851439bb236e4da28f252769b0d65177 2023-02-11 03:55:25,906 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=24, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/ea4735f297efbbfcc4cbc0bd3d6e42e0/.tmp/family2/e7348b782602480c9a597e465ab9b946 2023-02-11 03:55:25,909 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for e7348b782602480c9a597e465ab9b946 2023-02-11 03:55:25,909 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=24, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/621cd59809a16fd910172a9890d26263/.tmp/family2/4dd7fe8bbc3a4da88c569f143a7e6aac 2023-02-11 03:55:25,910 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/ea4735f297efbbfcc4cbc0bd3d6e42e0/.tmp/family1/48f103d11dc0460c87b57771c52929c1 as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/ea4735f297efbbfcc4cbc0bd3d6e42e0/family1/48f103d11dc0460c87b57771c52929c1 2023-02-11 03:55:25,910 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=24, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/186d87f306418fb25d295d824c1272fe/.tmp/family2/c18c054ee2504656bd96d4aab3a44c96 2023-02-11 03:55:25,913 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 4dd7fe8bbc3a4da88c569f143a7e6aac 2023-02-11 03:55:25,914 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 48f103d11dc0460c87b57771c52929c1 2023-02-11 03:55:25,914 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/621cd59809a16fd910172a9890d26263/.tmp/family1/fa6ce126bb87484288527c3ce84deb9c as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/621cd59809a16fd910172a9890d26263/family1/fa6ce126bb87484288527c3ce84deb9c 2023-02-11 03:55:25,914 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/ea4735f297efbbfcc4cbc0bd3d6e42e0/family1/48f103d11dc0460c87b57771c52929c1, entries=3, sequenceid=24, filesize=5.0 K 2023-02-11 03:55:25,914 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c18c054ee2504656bd96d4aab3a44c96 2023-02-11 03:55:25,915 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/186d87f306418fb25d295d824c1272fe/.tmp/family1/851439bb236e4da28f252769b0d65177 as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/186d87f306418fb25d295d824c1272fe/family1/851439bb236e4da28f252769b0d65177 2023-02-11 03:55:25,915 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/ea4735f297efbbfcc4cbc0bd3d6e42e0/.tmp/family2/e7348b782602480c9a597e465ab9b946 as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/ea4735f297efbbfcc4cbc0bd3d6e42e0/family2/e7348b782602480c9a597e465ab9b946 2023-02-11 03:55:25,918 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for fa6ce126bb87484288527c3ce84deb9c 2023-02-11 03:55:25,918 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/621cd59809a16fd910172a9890d26263/family1/fa6ce126bb87484288527c3ce84deb9c, entries=3, sequenceid=24, filesize=5.0 K 2023-02-11 03:55:25,918 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for e7348b782602480c9a597e465ab9b946 2023-02-11 03:55:25,918 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/ea4735f297efbbfcc4cbc0bd3d6e42e0/family2/e7348b782602480c9a597e465ab9b946, entries=1, sequenceid=24, filesize=5.0 K 2023-02-11 03:55:25,918 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/621cd59809a16fd910172a9890d26263/.tmp/family2/4dd7fe8bbc3a4da88c569f143a7e6aac as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/621cd59809a16fd910172a9890d26263/family2/4dd7fe8bbc3a4da88c569f143a7e6aac 2023-02-11 03:55:25,919 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 851439bb236e4da28f252769b0d65177 2023-02-11 03:55:25,919 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-2] regionserver.HRegion(2947): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for ea4735f297efbbfcc4cbc0bd3d6e42e0 in 1657ms, sequenceid=24, compaction requested=true 2023-02-11 03:55:25,919 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/186d87f306418fb25d295d824c1272fe/family1/851439bb236e4da28f252769b0d65177, entries=3, sequenceid=24, filesize=5.0 K 2023-02-11 03:55:25,919 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-2] regionserver.HRegion(2445): Flush status journal for ea4735f297efbbfcc4cbc0bd3d6e42e0: 2023-02-11 03:55:25,919 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,1,1676087711806.ea4735f297efbbfcc4cbc0bd3d6e42e0. 2023-02-11 03:55:25,919 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-02-11 03:55:25,920 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/186d87f306418fb25d295d824c1272fe/.tmp/family2/c18c054ee2504656bd96d4aab3a44c96 as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/186d87f306418fb25d295d824c1272fe/family2/c18c054ee2504656bd96d4aab3a44c96 2023-02-11 03:55:25,922 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 4dd7fe8bbc3a4da88c569f143a7e6aac 2023-02-11 03:55:25,922 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/621cd59809a16fd910172a9890d26263/family2/4dd7fe8bbc3a4da88c569f143a7e6aac, entries=1, sequenceid=24, filesize=5.0 K 2023-02-11 03:55:25,923 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for 621cd59809a16fd910172a9890d26263 in 1661ms, sequenceid=24, compaction requested=true 2023-02-11 03:55:25,923 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 621cd59809a16fd910172a9890d26263: 2023-02-11 03:55:25,923 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,2,1676087711806.621cd59809a16fd910172a9890d26263. 2023-02-11 03:55:25,923 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-02-11 03:55:25,923 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c18c054ee2504656bd96d4aab3a44c96 2023-02-11 03:55:25,923 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/186d87f306418fb25d295d824c1272fe/family2/c18c054ee2504656bd96d4aab3a44c96, entries=1, sequenceid=24, filesize=5.0 K 2023-02-11 03:55:25,924 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-1] regionserver.HRegion(2947): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for 186d87f306418fb25d295d824c1272fe in 1662ms, sequenceid=24, compaction requested=true 2023-02-11 03:55:25,924 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-1] regionserver.HRegion(2445): Flush status journal for 186d87f306418fb25d295d824c1272fe: 2023-02-11 03:55:25,924 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,,1676087711806.186d87f306418fb25d295d824c1272fe. 2023-02-11 03:55:25,924 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-02-11 03:55:25,924 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-02-11 03:55:25,924 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:55:25,924 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testCompactionWithHFileLink' locally acquired 2023-02-11 03:55:25,924 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37011,1676087618428' joining acquired barrier for procedure (testCompactionWithHFileLink) in zk 2023-02-11 03:55:25,925 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:55:25,925 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-11 03:55:25,925 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:55:25,925 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:55:25,925 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:55:25,926 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-11 03:55:25,926 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testCompactionWithHFileLink' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 03:55:25,926 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:55:25,926 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:55:25,926 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-11 03:55:25,926 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:55:25,926 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:55:25,927 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37011,1676087618428' joining acquired barrier for procedure 'testCompactionWithHFileLink' on coordinator 2023-02-11 03:55:25,927 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7f9e15be[Count = 0] remaining members to acquire global barrier 2023-02-11 03:55:25,927 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testCompactionWithHFileLink' starting 'in-barrier' execution. 2023-02-11 03:55:25,927 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-11 03:55:25,928 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-11 03:55:25,928 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-11 03:55:25,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-11 03:55:25,928 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testCompactionWithHFileLink' received 'reached' from coordinator. 2023-02-11 03:55:25,928 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testCompactionWithHFileLink' locally completed 2023-02-11 03:55:25,928 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testCompactionWithHFileLink' completed for member 'jenkins-hbase4.apache.org,37011,1676087618428' in zk 2023-02-11 03:55:25,928 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:55:25,928 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 03:55:25,935 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testCompactionWithHFileLink' has notified controller of completion 2023-02-11 03:55:25,935 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:55:25,935 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:55:25,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:55:25,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:55:25,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:55:25,935 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testCompactionWithHFileLink' completed. 2023-02-11 03:55:25,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:55:25,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:55:25,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-11 03:55:25,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:55:25,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:55:25,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-11 03:55:25,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:55:25,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testCompactionWithHFileLink' member 'jenkins-hbase4.apache.org,37011,1676087618428': 2023-02-11 03:55:25,937 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37011,1676087618428' released barrier for procedure'testCompactionWithHFileLink', counting down latch. Waiting for 0 more 2023-02-11 03:55:25,937 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testCompactionWithHFileLink' execution completed 2023-02-11 03:55:25,937 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 03:55:25,937 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 03:55:25,937 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testCompactionWithHFileLink 2023-02-11 03:55:25,937 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testCompactionWithHFileLinkincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 03:55:25,938 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-11 03:55:25,938 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-11 03:55:25,938 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-11 03:55:25,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-11 03:55:25,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:55:25,938 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:55:25,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:55:25,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-11 03:55:25,939 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:55:25,939 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:55:25,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:55:25,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:55:25,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-11 03:55:25,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-11 03:55:25,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:55:25,940 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:55:25,940 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-11 03:55:25,940 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:55:25,940 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:55:25,940 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-11 03:55:25,940 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:55:25,941 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:55:25,941 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:55:25,941 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-02-11 03:55:25,941 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:55:25,941 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testCompactionWithHFileLink' 2023-02-11 03:55:25,942 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 03:55:25,941 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-02-11 03:55:25,941 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:55:25,941 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:55:25,942 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:55:25,942 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink because node does not exist (not an error) 2023-02-11 03:55:25,942 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-11 03:55:25,942 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testCompactionWithHFileLink'' to complete. (max 300000 ms per retry) 2023-02-11 03:55:25,942 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 03:55:25,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:55:25,942 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-11 03:55:25,942 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-11 03:55:25,942 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:55:25,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:55:26,042 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 03:55:26,043 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 03:55:26,044 DEBUG [Listener at localhost/46205] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-02-11 03:55:26,044 DEBUG [Listener at localhost/46205] regionserver.HStore(1912): 621cd59809a16fd910172a9890d26263/family1 is initiating major compaction (all files) 2023-02-11 03:55:26,044 INFO [Listener at localhost/46205] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-11 03:55:26,044 INFO [Listener at localhost/46205] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-11 03:55:26,044 INFO [Listener at localhost/46205] regionserver.HRegion(2258): Starting compaction of 621cd59809a16fd910172a9890d26263/family1 in testCompactionWithHFileLink,2,1676087711806.621cd59809a16fd910172a9890d26263. 2023-02-11 03:55:26,044 INFO [Listener at localhost/46205] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/621cd59809a16fd910172a9890d26263/family1/a54c49bae3b34318920fc4197db7ae61, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/621cd59809a16fd910172a9890d26263/family1/051a49ec8101444f8e462520d66fa021, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/621cd59809a16fd910172a9890d26263/family1/3cb4dc3b66114935967cd14111320066, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/621cd59809a16fd910172a9890d26263/family1/42ea1ea361834322b07cb7fa95ee893f, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/621cd59809a16fd910172a9890d26263/family1/fa6ce126bb87484288527c3ce84deb9c] into tmpdir=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/621cd59809a16fd910172a9890d26263/.tmp, totalSize=26.7 K 2023-02-11 03:55:26,045 DEBUG [Listener at localhost/46205] compactions.Compactor(207): Compacting a54c49bae3b34318920fc4197db7ae61, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1676087712922 2023-02-11 03:55:26,045 DEBUG [Listener at localhost/46205] compactions.Compactor(207): Compacting 051a49ec8101444f8e462520d66fa021, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1676087714701 2023-02-11 03:55:26,046 DEBUG [Listener at localhost/46205] compactions.Compactor(207): Compacting 3cb4dc3b66114935967cd14111320066, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1676087716494 2023-02-11 03:55:26,046 DEBUG [Listener at localhost/46205] compactions.Compactor(207): Compacting 42ea1ea361834322b07cb7fa95ee893f, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1676087718286 2023-02-11 03:55:26,046 DEBUG [Listener at localhost/46205] compactions.Compactor(207): Compacting fa6ce126bb87484288527c3ce84deb9c, keycount=3, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=24, earliestPutTs=9223372036854775807 2023-02-11 03:55:26,058 INFO [Listener at localhost/46205] throttle.PressureAwareThroughputController(145): 621cd59809a16fd910172a9890d26263#family1#compaction#107 average throughput is 0.57 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-11 03:55:26,466 INFO [Listener at localhost/46205] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c2023021172810e71b3c3497c8cf0551a5ae35902_del 2023-02-11 03:55:26,466 INFO [Listener at localhost/46205] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c2023021172810e71b3c3497c8cf0551a5ae35902_del to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c2023021172810e71b3c3497c8cf0551a5ae35902_del 2023-02-11 03:55:27,279 INFO [Listener at localhost/46205] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 6b911754855f4041855ad51000f60d9e 2023-02-11 03:55:27,280 DEBUG [Listener at localhost/46205] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/621cd59809a16fd910172a9890d26263/.tmp/family1/6b911754855f4041855ad51000f60d9e as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/621cd59809a16fd910172a9890d26263/family1/6b911754855f4041855ad51000f60d9e 2023-02-11 03:55:27,284 INFO [Listener at localhost/46205] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 6b911754855f4041855ad51000f60d9e 2023-02-11 03:55:27,284 INFO [Listener at localhost/46205] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 621cd59809a16fd910172a9890d26263/family1 of 621cd59809a16fd910172a9890d26263 into 6b911754855f4041855ad51000f60d9e(size=6.9 K), total size for store is 6.9 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-02-11 03:55:27,285 DEBUG [Listener at localhost/46205] regionserver.HRegion(2288): Compaction status journal for 621cd59809a16fd910172a9890d26263: 2023-02-11 03:55:27,285 DEBUG [Listener at localhost/46205] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-02-11 03:55:27,285 DEBUG [Listener at localhost/46205] regionserver.HStore(1912): 621cd59809a16fd910172a9890d26263/family2 is initiating major compaction (all files) 2023-02-11 03:55:27,285 INFO [Listener at localhost/46205] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-11 03:55:27,285 INFO [Listener at localhost/46205] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-11 03:55:27,286 INFO [Listener at localhost/46205] regionserver.HRegion(2258): Starting compaction of 621cd59809a16fd910172a9890d26263/family2 in testCompactionWithHFileLink,2,1676087711806.621cd59809a16fd910172a9890d26263. 2023-02-11 03:55:27,286 INFO [Listener at localhost/46205] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/621cd59809a16fd910172a9890d26263/family2/774f5f8b4d2e463f983d817bdedf591e, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/621cd59809a16fd910172a9890d26263/family2/e000a979eed34f31ba1476e0d7c949da, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/621cd59809a16fd910172a9890d26263/family2/87ccd53b473f47b6887f864a901e3b7c, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/621cd59809a16fd910172a9890d26263/family2/32530f87de8648c0ab8a83636ca1711f, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/621cd59809a16fd910172a9890d26263/family2/4dd7fe8bbc3a4da88c569f143a7e6aac] into tmpdir=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/621cd59809a16fd910172a9890d26263/.tmp, totalSize=25.4 K 2023-02-11 03:55:27,286 DEBUG [Listener at localhost/46205] compactions.Compactor(207): Compacting 774f5f8b4d2e463f983d817bdedf591e, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1676087712922 2023-02-11 03:55:27,286 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/621cd59809a16fd910172a9890d26263/family1/a54c49bae3b34318920fc4197db7ae61, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/621cd59809a16fd910172a9890d26263/family1/051a49ec8101444f8e462520d66fa021, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/621cd59809a16fd910172a9890d26263/family1/3cb4dc3b66114935967cd14111320066, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/621cd59809a16fd910172a9890d26263/family1/42ea1ea361834322b07cb7fa95ee893f, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/621cd59809a16fd910172a9890d26263/family1/fa6ce126bb87484288527c3ce84deb9c] to archive 2023-02-11 03:55:27,287 DEBUG [Listener at localhost/46205] compactions.Compactor(207): Compacting e000a979eed34f31ba1476e0d7c949da, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1676087714701 2023-02-11 03:55:27,287 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-11 03:55:27,287 DEBUG [Listener at localhost/46205] compactions.Compactor(207): Compacting 87ccd53b473f47b6887f864a901e3b7c, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1676087716494 2023-02-11 03:55:27,287 DEBUG [Listener at localhost/46205] compactions.Compactor(207): Compacting 32530f87de8648c0ab8a83636ca1711f, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1676087718286 2023-02-11 03:55:27,288 DEBUG [Listener at localhost/46205] compactions.Compactor(207): Compacting 4dd7fe8bbc3a4da88c569f143a7e6aac, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=24, earliestPutTs=9223372036854775807 2023-02-11 03:55:27,288 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/621cd59809a16fd910172a9890d26263/family1/a54c49bae3b34318920fc4197db7ae61 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testCompactionWithHFileLink/621cd59809a16fd910172a9890d26263/family1/a54c49bae3b34318920fc4197db7ae61 2023-02-11 03:55:27,290 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/621cd59809a16fd910172a9890d26263/family1/051a49ec8101444f8e462520d66fa021 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testCompactionWithHFileLink/621cd59809a16fd910172a9890d26263/family1/051a49ec8101444f8e462520d66fa021 2023-02-11 03:55:27,291 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/621cd59809a16fd910172a9890d26263/family1/3cb4dc3b66114935967cd14111320066 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testCompactionWithHFileLink/621cd59809a16fd910172a9890d26263/family1/3cb4dc3b66114935967cd14111320066 2023-02-11 03:55:27,292 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/621cd59809a16fd910172a9890d26263/family1/42ea1ea361834322b07cb7fa95ee893f to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testCompactionWithHFileLink/621cd59809a16fd910172a9890d26263/family1/42ea1ea361834322b07cb7fa95ee893f 2023-02-11 03:55:27,293 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/621cd59809a16fd910172a9890d26263/family1/fa6ce126bb87484288527c3ce84deb9c to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testCompactionWithHFileLink/621cd59809a16fd910172a9890d26263/family1/fa6ce126bb87484288527c3ce84deb9c 2023-02-11 03:55:27,449 INFO [regionserver/jenkins-hbase4:0.Chore.1] hbase.ScheduledChore(142): Chore: CompactedHFilesCleaner missed its start time 2023-02-11 03:55:27,449 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-02-11 03:55:27,449 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x369d21bb to 127.0.0.1:53383 2023-02-11 03:55:27,449 DEBUG [Finalizer] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-11 03:55:27,452 INFO [Listener at localhost/46205] throttle.PressureAwareThroughputController(145): 621cd59809a16fd910172a9890d26263#family2#compaction#108 average throughput is 0.01 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-11 03:55:27,861 INFO [Listener at localhost/46205] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c20230211d2516239b51e4745bb93c4eebc72cef9_del 2023-02-11 03:55:27,861 INFO [Listener at localhost/46205] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230211d2516239b51e4745bb93c4eebc72cef9_del to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230211d2516239b51e4745bb93c4eebc72cef9_del 2023-02-11 03:55:28,674 INFO [Listener at localhost/46205] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 8e6e96733be0435d8cd67d27596328ca 2023-02-11 03:55:28,675 DEBUG [Listener at localhost/46205] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/621cd59809a16fd910172a9890d26263/.tmp/family2/8e6e96733be0435d8cd67d27596328ca as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/621cd59809a16fd910172a9890d26263/family2/8e6e96733be0435d8cd67d27596328ca 2023-02-11 03:55:28,678 INFO [Listener at localhost/46205] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 8e6e96733be0435d8cd67d27596328ca 2023-02-11 03:55:28,679 INFO [Listener at localhost/46205] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 621cd59809a16fd910172a9890d26263/family2 of 621cd59809a16fd910172a9890d26263 into 8e6e96733be0435d8cd67d27596328ca(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-02-11 03:55:28,679 DEBUG [Listener at localhost/46205] regionserver.HRegion(2288): Compaction status journal for 621cd59809a16fd910172a9890d26263: 2023-02-11 03:55:28,679 DEBUG [Listener at localhost/46205] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-02-11 03:55:28,679 DEBUG [Listener at localhost/46205] regionserver.HStore(1912): 186d87f306418fb25d295d824c1272fe/family1 is initiating major compaction (all files) 2023-02-11 03:55:28,680 INFO [Listener at localhost/46205] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-11 03:55:28,680 INFO [Listener at localhost/46205] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-11 03:55:28,680 INFO [Listener at localhost/46205] regionserver.HRegion(2258): Starting compaction of 186d87f306418fb25d295d824c1272fe/family1 in testCompactionWithHFileLink,,1676087711806.186d87f306418fb25d295d824c1272fe. 2023-02-11 03:55:28,680 INFO [Listener at localhost/46205] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/186d87f306418fb25d295d824c1272fe/family1/2e6dcfa9c78543fcb2273f9d67b88e36, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/186d87f306418fb25d295d824c1272fe/family1/4e9479861f4540a6b8a67dee5f4b8c2c, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/186d87f306418fb25d295d824c1272fe/family1/55bf25ed3b544fe9969ac6430989cefc, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/186d87f306418fb25d295d824c1272fe/family1/1363e2a6dd1f49e0a9d550f758d2bcac, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/186d87f306418fb25d295d824c1272fe/family1/851439bb236e4da28f252769b0d65177] into tmpdir=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/186d87f306418fb25d295d824c1272fe/.tmp, totalSize=26.7 K 2023-02-11 03:55:28,680 DEBUG [Listener at localhost/46205] compactions.Compactor(207): Compacting 2e6dcfa9c78543fcb2273f9d67b88e36, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1676087712920 2023-02-11 03:55:28,681 DEBUG [Listener at localhost/46205] compactions.Compactor(207): Compacting 4e9479861f4540a6b8a67dee5f4b8c2c, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1676087714701 2023-02-11 03:55:28,681 DEBUG [Listener at localhost/46205] compactions.Compactor(207): Compacting 55bf25ed3b544fe9969ac6430989cefc, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1676087716494 2023-02-11 03:55:28,681 DEBUG [Listener at localhost/46205] compactions.Compactor(207): Compacting 1363e2a6dd1f49e0a9d550f758d2bcac, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1676087718286 2023-02-11 03:55:28,681 DEBUG [Listener at localhost/46205] compactions.Compactor(207): Compacting 851439bb236e4da28f252769b0d65177, keycount=3, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=24, earliestPutTs=9223372036854775807 2023-02-11 03:55:28,691 INFO [Listener at localhost/46205] throttle.PressureAwareThroughputController(145): 186d87f306418fb25d295d824c1272fe#family1#compaction#109 average throughput is 0.85 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-11 03:55:28,756 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/621cd59809a16fd910172a9890d26263/family2/774f5f8b4d2e463f983d817bdedf591e, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/621cd59809a16fd910172a9890d26263/family2/e000a979eed34f31ba1476e0d7c949da, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/621cd59809a16fd910172a9890d26263/family2/87ccd53b473f47b6887f864a901e3b7c, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/621cd59809a16fd910172a9890d26263/family2/32530f87de8648c0ab8a83636ca1711f, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/621cd59809a16fd910172a9890d26263/family2/4dd7fe8bbc3a4da88c569f143a7e6aac] to archive 2023-02-11 03:55:28,757 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-11 03:55:28,758 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/621cd59809a16fd910172a9890d26263/family2/774f5f8b4d2e463f983d817bdedf591e to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testCompactionWithHFileLink/621cd59809a16fd910172a9890d26263/family2/774f5f8b4d2e463f983d817bdedf591e 2023-02-11 03:55:28,759 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/621cd59809a16fd910172a9890d26263/family2/e000a979eed34f31ba1476e0d7c949da to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testCompactionWithHFileLink/621cd59809a16fd910172a9890d26263/family2/e000a979eed34f31ba1476e0d7c949da 2023-02-11 03:55:28,760 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/621cd59809a16fd910172a9890d26263/family2/87ccd53b473f47b6887f864a901e3b7c to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testCompactionWithHFileLink/621cd59809a16fd910172a9890d26263/family2/87ccd53b473f47b6887f864a901e3b7c 2023-02-11 03:55:28,761 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/621cd59809a16fd910172a9890d26263/family2/32530f87de8648c0ab8a83636ca1711f to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testCompactionWithHFileLink/621cd59809a16fd910172a9890d26263/family2/32530f87de8648c0ab8a83636ca1711f 2023-02-11 03:55:28,762 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/621cd59809a16fd910172a9890d26263/family2/4dd7fe8bbc3a4da88c569f143a7e6aac to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testCompactionWithHFileLink/621cd59809a16fd910172a9890d26263/family2/4dd7fe8bbc3a4da88c569f143a7e6aac 2023-02-11 03:55:29,099 INFO [Listener at localhost/46205] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e20230211bb7a69e13f50443eb5a9c83d8fe1e705_del 2023-02-11 03:55:29,099 INFO [Listener at localhost/46205] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230211bb7a69e13f50443eb5a9c83d8fe1e705_del to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230211bb7a69e13f50443eb5a9c83d8fe1e705_del 2023-02-11 03:55:29,912 INFO [Listener at localhost/46205] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 33bb864dff41474f9562086b84b9018e 2023-02-11 03:55:29,913 DEBUG [Listener at localhost/46205] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/186d87f306418fb25d295d824c1272fe/.tmp/family1/33bb864dff41474f9562086b84b9018e as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/186d87f306418fb25d295d824c1272fe/family1/33bb864dff41474f9562086b84b9018e 2023-02-11 03:55:29,916 INFO [Listener at localhost/46205] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 33bb864dff41474f9562086b84b9018e 2023-02-11 03:55:29,917 INFO [Listener at localhost/46205] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 186d87f306418fb25d295d824c1272fe/family1 of 186d87f306418fb25d295d824c1272fe into 33bb864dff41474f9562086b84b9018e(size=6.9 K), total size for store is 6.9 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-02-11 03:55:29,917 DEBUG [Listener at localhost/46205] regionserver.HRegion(2288): Compaction status journal for 186d87f306418fb25d295d824c1272fe: 2023-02-11 03:55:29,917 DEBUG [Listener at localhost/46205] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-02-11 03:55:29,917 DEBUG [Listener at localhost/46205] regionserver.HStore(1912): 186d87f306418fb25d295d824c1272fe/family2 is initiating major compaction (all files) 2023-02-11 03:55:29,918 INFO [Listener at localhost/46205] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-11 03:55:29,918 INFO [Listener at localhost/46205] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-11 03:55:29,918 INFO [Listener at localhost/46205] regionserver.HRegion(2258): Starting compaction of 186d87f306418fb25d295d824c1272fe/family2 in testCompactionWithHFileLink,,1676087711806.186d87f306418fb25d295d824c1272fe. 2023-02-11 03:55:29,918 INFO [Listener at localhost/46205] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/186d87f306418fb25d295d824c1272fe/family2/48173b765b124663a97b2f43b90e8a88, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/186d87f306418fb25d295d824c1272fe/family2/039e3e0c12c742f29c5ea86a66e98c44, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/186d87f306418fb25d295d824c1272fe/family2/14bfe13c870f426d986ef125637ce8f4, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/186d87f306418fb25d295d824c1272fe/family2/203bb908b48a4908b252cec8e0b7c5e7, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/186d87f306418fb25d295d824c1272fe/family2/c18c054ee2504656bd96d4aab3a44c96] into tmpdir=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/186d87f306418fb25d295d824c1272fe/.tmp, totalSize=25.4 K 2023-02-11 03:55:29,918 DEBUG [Listener at localhost/46205] compactions.Compactor(207): Compacting 48173b765b124663a97b2f43b90e8a88, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1676087712920 2023-02-11 03:55:29,919 DEBUG [Listener at localhost/46205] compactions.Compactor(207): Compacting 039e3e0c12c742f29c5ea86a66e98c44, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1676087714701 2023-02-11 03:55:29,919 DEBUG [Listener at localhost/46205] compactions.Compactor(207): Compacting 14bfe13c870f426d986ef125637ce8f4, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1676087716494 2023-02-11 03:55:29,919 DEBUG [Listener at localhost/46205] compactions.Compactor(207): Compacting 203bb908b48a4908b252cec8e0b7c5e7, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1676087718286 2023-02-11 03:55:29,920 DEBUG [Listener at localhost/46205] compactions.Compactor(207): Compacting c18c054ee2504656bd96d4aab3a44c96, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=24, earliestPutTs=9223372036854775807 2023-02-11 03:55:29,930 INFO [Listener at localhost/46205] throttle.PressureAwareThroughputController(145): 186d87f306418fb25d295d824c1272fe#family2#compaction#110 average throughput is 0.53 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-11 03:55:29,949 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/186d87f306418fb25d295d824c1272fe/family1/2e6dcfa9c78543fcb2273f9d67b88e36, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/186d87f306418fb25d295d824c1272fe/family1/4e9479861f4540a6b8a67dee5f4b8c2c, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/186d87f306418fb25d295d824c1272fe/family1/55bf25ed3b544fe9969ac6430989cefc, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/186d87f306418fb25d295d824c1272fe/family1/1363e2a6dd1f49e0a9d550f758d2bcac, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/186d87f306418fb25d295d824c1272fe/family1/851439bb236e4da28f252769b0d65177] to archive 2023-02-11 03:55:29,950 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-11 03:55:29,951 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/186d87f306418fb25d295d824c1272fe/family1/2e6dcfa9c78543fcb2273f9d67b88e36 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testCompactionWithHFileLink/186d87f306418fb25d295d824c1272fe/family1/2e6dcfa9c78543fcb2273f9d67b88e36 2023-02-11 03:55:29,952 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/186d87f306418fb25d295d824c1272fe/family1/4e9479861f4540a6b8a67dee5f4b8c2c to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testCompactionWithHFileLink/186d87f306418fb25d295d824c1272fe/family1/4e9479861f4540a6b8a67dee5f4b8c2c 2023-02-11 03:55:29,953 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/186d87f306418fb25d295d824c1272fe/family1/55bf25ed3b544fe9969ac6430989cefc to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testCompactionWithHFileLink/186d87f306418fb25d295d824c1272fe/family1/55bf25ed3b544fe9969ac6430989cefc 2023-02-11 03:55:29,954 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/186d87f306418fb25d295d824c1272fe/family1/1363e2a6dd1f49e0a9d550f758d2bcac to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testCompactionWithHFileLink/186d87f306418fb25d295d824c1272fe/family1/1363e2a6dd1f49e0a9d550f758d2bcac 2023-02-11 03:55:29,955 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/186d87f306418fb25d295d824c1272fe/family1/851439bb236e4da28f252769b0d65177 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testCompactionWithHFileLink/186d87f306418fb25d295d824c1272fe/family1/851439bb236e4da28f252769b0d65177 2023-02-11 03:55:30,338 INFO [Listener at localhost/46205] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e20230211bf6b892b644c4f1fab3514747a0eaf2a_del 2023-02-11 03:55:30,339 INFO [Listener at localhost/46205] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230211bf6b892b644c4f1fab3514747a0eaf2a_del to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230211bf6b892b644c4f1fab3514747a0eaf2a_del 2023-02-11 03:55:31,152 INFO [Listener at localhost/46205] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 6046f3d6ed634b868e201512cc6356ce 2023-02-11 03:55:31,152 DEBUG [Listener at localhost/46205] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/186d87f306418fb25d295d824c1272fe/.tmp/family2/6046f3d6ed634b868e201512cc6356ce as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/186d87f306418fb25d295d824c1272fe/family2/6046f3d6ed634b868e201512cc6356ce 2023-02-11 03:55:31,156 INFO [Listener at localhost/46205] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 6046f3d6ed634b868e201512cc6356ce 2023-02-11 03:55:31,156 INFO [Listener at localhost/46205] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 186d87f306418fb25d295d824c1272fe/family2 of 186d87f306418fb25d295d824c1272fe into 6046f3d6ed634b868e201512cc6356ce(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-02-11 03:55:31,157 DEBUG [Listener at localhost/46205] regionserver.HRegion(2288): Compaction status journal for 186d87f306418fb25d295d824c1272fe: 2023-02-11 03:55:31,157 DEBUG [Listener at localhost/46205] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-02-11 03:55:31,157 DEBUG [Listener at localhost/46205] regionserver.HStore(1912): ea4735f297efbbfcc4cbc0bd3d6e42e0/family1 is initiating major compaction (all files) 2023-02-11 03:55:31,157 INFO [Listener at localhost/46205] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-11 03:55:31,157 INFO [Listener at localhost/46205] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-11 03:55:31,158 INFO [Listener at localhost/46205] regionserver.HRegion(2258): Starting compaction of ea4735f297efbbfcc4cbc0bd3d6e42e0/family1 in testCompactionWithHFileLink,1,1676087711806.ea4735f297efbbfcc4cbc0bd3d6e42e0. 2023-02-11 03:55:31,158 INFO [Listener at localhost/46205] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/ea4735f297efbbfcc4cbc0bd3d6e42e0/family1/92743cf8525b4b5d877b358e975a3bec, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/ea4735f297efbbfcc4cbc0bd3d6e42e0/family1/0a642f51d6d94a79b4c5663961dee5c0, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/ea4735f297efbbfcc4cbc0bd3d6e42e0/family1/c504159ab71244858e1768f8eb16927b, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/ea4735f297efbbfcc4cbc0bd3d6e42e0/family1/3088ce87ec264e38859de9f824a4ab99, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/ea4735f297efbbfcc4cbc0bd3d6e42e0/family1/48f103d11dc0460c87b57771c52929c1] into tmpdir=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/ea4735f297efbbfcc4cbc0bd3d6e42e0/.tmp, totalSize=26.7 K 2023-02-11 03:55:31,158 DEBUG [Listener at localhost/46205] compactions.Compactor(207): Compacting 92743cf8525b4b5d877b358e975a3bec, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1676087712921 2023-02-11 03:55:31,158 DEBUG [Listener at localhost/46205] compactions.Compactor(207): Compacting 0a642f51d6d94a79b4c5663961dee5c0, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1676087714701 2023-02-11 03:55:31,159 DEBUG [Listener at localhost/46205] compactions.Compactor(207): Compacting c504159ab71244858e1768f8eb16927b, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1676087716494 2023-02-11 03:55:31,159 DEBUG [Listener at localhost/46205] compactions.Compactor(207): Compacting 3088ce87ec264e38859de9f824a4ab99, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1676087718286 2023-02-11 03:55:31,159 DEBUG [Listener at localhost/46205] compactions.Compactor(207): Compacting 48f103d11dc0460c87b57771c52929c1, keycount=3, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=24, earliestPutTs=9223372036854775807 2023-02-11 03:55:31,169 INFO [Listener at localhost/46205] throttle.PressureAwareThroughputController(145): ea4735f297efbbfcc4cbc0bd3d6e42e0#family1#compaction#111 average throughput is 0.57 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-11 03:55:31,252 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/186d87f306418fb25d295d824c1272fe/family2/48173b765b124663a97b2f43b90e8a88, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/186d87f306418fb25d295d824c1272fe/family2/039e3e0c12c742f29c5ea86a66e98c44, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/186d87f306418fb25d295d824c1272fe/family2/14bfe13c870f426d986ef125637ce8f4, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/186d87f306418fb25d295d824c1272fe/family2/203bb908b48a4908b252cec8e0b7c5e7, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/186d87f306418fb25d295d824c1272fe/family2/c18c054ee2504656bd96d4aab3a44c96] to archive 2023-02-11 03:55:31,253 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-11 03:55:31,254 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/186d87f306418fb25d295d824c1272fe/family2/48173b765b124663a97b2f43b90e8a88 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testCompactionWithHFileLink/186d87f306418fb25d295d824c1272fe/family2/48173b765b124663a97b2f43b90e8a88 2023-02-11 03:55:31,255 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/186d87f306418fb25d295d824c1272fe/family2/039e3e0c12c742f29c5ea86a66e98c44 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testCompactionWithHFileLink/186d87f306418fb25d295d824c1272fe/family2/039e3e0c12c742f29c5ea86a66e98c44 2023-02-11 03:55:31,256 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/186d87f306418fb25d295d824c1272fe/family2/14bfe13c870f426d986ef125637ce8f4 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testCompactionWithHFileLink/186d87f306418fb25d295d824c1272fe/family2/14bfe13c870f426d986ef125637ce8f4 2023-02-11 03:55:31,257 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/186d87f306418fb25d295d824c1272fe/family2/203bb908b48a4908b252cec8e0b7c5e7 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testCompactionWithHFileLink/186d87f306418fb25d295d824c1272fe/family2/203bb908b48a4908b252cec8e0b7c5e7 2023-02-11 03:55:31,258 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/186d87f306418fb25d295d824c1272fe/family2/c18c054ee2504656bd96d4aab3a44c96 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testCompactionWithHFileLink/186d87f306418fb25d295d824c1272fe/family2/c18c054ee2504656bd96d4aab3a44c96 2023-02-11 03:55:31,576 INFO [Listener at localhost/46205] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b20230211a385f41cb4df40b08d76951be11968e4_del 2023-02-11 03:55:31,576 INFO [Listener at localhost/46205] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230211a385f41cb4df40b08d76951be11968e4_del to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230211a385f41cb4df40b08d76951be11968e4_del 2023-02-11 03:55:32,389 INFO [Listener at localhost/46205] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 553adfdab49e4996b9384047185ed50a 2023-02-11 03:55:32,390 DEBUG [Listener at localhost/46205] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/ea4735f297efbbfcc4cbc0bd3d6e42e0/.tmp/family1/553adfdab49e4996b9384047185ed50a as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/ea4735f297efbbfcc4cbc0bd3d6e42e0/family1/553adfdab49e4996b9384047185ed50a 2023-02-11 03:55:32,394 INFO [Listener at localhost/46205] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 553adfdab49e4996b9384047185ed50a 2023-02-11 03:55:32,395 INFO [Listener at localhost/46205] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in ea4735f297efbbfcc4cbc0bd3d6e42e0/family1 of ea4735f297efbbfcc4cbc0bd3d6e42e0 into 553adfdab49e4996b9384047185ed50a(size=6.9 K), total size for store is 6.9 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-02-11 03:55:32,395 DEBUG [Listener at localhost/46205] regionserver.HRegion(2288): Compaction status journal for ea4735f297efbbfcc4cbc0bd3d6e42e0: 2023-02-11 03:55:32,395 DEBUG [Listener at localhost/46205] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-02-11 03:55:32,395 DEBUG [Listener at localhost/46205] regionserver.HStore(1912): ea4735f297efbbfcc4cbc0bd3d6e42e0/family2 is initiating major compaction (all files) 2023-02-11 03:55:32,395 INFO [Listener at localhost/46205] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-11 03:55:32,395 INFO [Listener at localhost/46205] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-11 03:55:32,395 INFO [Listener at localhost/46205] regionserver.HRegion(2258): Starting compaction of ea4735f297efbbfcc4cbc0bd3d6e42e0/family2 in testCompactionWithHFileLink,1,1676087711806.ea4735f297efbbfcc4cbc0bd3d6e42e0. 2023-02-11 03:55:32,395 INFO [Listener at localhost/46205] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/ea4735f297efbbfcc4cbc0bd3d6e42e0/family2/7141c179434b4a0f816573a88da1caf7, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/ea4735f297efbbfcc4cbc0bd3d6e42e0/family2/045314dc19574fe9979fbedc8e14acf8, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/ea4735f297efbbfcc4cbc0bd3d6e42e0/family2/a77823d8bc254eb5b1130f536fa90187, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/ea4735f297efbbfcc4cbc0bd3d6e42e0/family2/5b41b5f362864186bb9f20144e3bc4c5, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/ea4735f297efbbfcc4cbc0bd3d6e42e0/family2/e7348b782602480c9a597e465ab9b946] into tmpdir=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/ea4735f297efbbfcc4cbc0bd3d6e42e0/.tmp, totalSize=25.4 K 2023-02-11 03:55:32,396 DEBUG [Listener at localhost/46205] compactions.Compactor(207): Compacting 7141c179434b4a0f816573a88da1caf7, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1676087712921 2023-02-11 03:55:32,396 DEBUG [Listener at localhost/46205] compactions.Compactor(207): Compacting 045314dc19574fe9979fbedc8e14acf8, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1676087714701 2023-02-11 03:55:32,396 DEBUG [Listener at localhost/46205] compactions.Compactor(207): Compacting a77823d8bc254eb5b1130f536fa90187, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1676087716494 2023-02-11 03:55:32,397 DEBUG [Listener at localhost/46205] compactions.Compactor(207): Compacting 5b41b5f362864186bb9f20144e3bc4c5, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1676087718286 2023-02-11 03:55:32,397 DEBUG [Listener at localhost/46205] compactions.Compactor(207): Compacting e7348b782602480c9a597e465ab9b946, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=24, earliestPutTs=9223372036854775807 2023-02-11 03:55:32,408 INFO [Listener at localhost/46205] throttle.PressureAwareThroughputController(145): ea4735f297efbbfcc4cbc0bd3d6e42e0#family2#compaction#112 average throughput is 0.35 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-11 03:55:32,449 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/ea4735f297efbbfcc4cbc0bd3d6e42e0/family1/92743cf8525b4b5d877b358e975a3bec, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/ea4735f297efbbfcc4cbc0bd3d6e42e0/family1/0a642f51d6d94a79b4c5663961dee5c0, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/ea4735f297efbbfcc4cbc0bd3d6e42e0/family1/c504159ab71244858e1768f8eb16927b, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/ea4735f297efbbfcc4cbc0bd3d6e42e0/family1/3088ce87ec264e38859de9f824a4ab99, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/ea4735f297efbbfcc4cbc0bd3d6e42e0/family1/48f103d11dc0460c87b57771c52929c1] to archive 2023-02-11 03:55:32,450 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-11 03:55:32,452 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/ea4735f297efbbfcc4cbc0bd3d6e42e0/family1/92743cf8525b4b5d877b358e975a3bec to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testCompactionWithHFileLink/ea4735f297efbbfcc4cbc0bd3d6e42e0/family1/92743cf8525b4b5d877b358e975a3bec 2023-02-11 03:55:32,453 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/ea4735f297efbbfcc4cbc0bd3d6e42e0/family1/0a642f51d6d94a79b4c5663961dee5c0 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testCompactionWithHFileLink/ea4735f297efbbfcc4cbc0bd3d6e42e0/family1/0a642f51d6d94a79b4c5663961dee5c0 2023-02-11 03:55:32,454 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/ea4735f297efbbfcc4cbc0bd3d6e42e0/family1/c504159ab71244858e1768f8eb16927b to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testCompactionWithHFileLink/ea4735f297efbbfcc4cbc0bd3d6e42e0/family1/c504159ab71244858e1768f8eb16927b 2023-02-11 03:55:32,455 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/ea4735f297efbbfcc4cbc0bd3d6e42e0/family1/3088ce87ec264e38859de9f824a4ab99 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testCompactionWithHFileLink/ea4735f297efbbfcc4cbc0bd3d6e42e0/family1/3088ce87ec264e38859de9f824a4ab99 2023-02-11 03:55:32,456 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/ea4735f297efbbfcc4cbc0bd3d6e42e0/family1/48f103d11dc0460c87b57771c52929c1 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testCompactionWithHFileLink/ea4735f297efbbfcc4cbc0bd3d6e42e0/family1/48f103d11dc0460c87b57771c52929c1 2023-02-11 03:55:32,817 INFO [Listener at localhost/46205] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b20230211a0ddb0b1c2124b71a00870343e6a77d3_del 2023-02-11 03:55:32,818 INFO [Listener at localhost/46205] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230211a0ddb0b1c2124b71a00870343e6a77d3_del to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230211a0ddb0b1c2124b71a00870343e6a77d3_del 2023-02-11 03:55:33,630 INFO [Listener at localhost/46205] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for bccb725e01334e4b8e122c684ffc0e2e 2023-02-11 03:55:33,631 DEBUG [Listener at localhost/46205] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/ea4735f297efbbfcc4cbc0bd3d6e42e0/.tmp/family2/bccb725e01334e4b8e122c684ffc0e2e as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/ea4735f297efbbfcc4cbc0bd3d6e42e0/family2/bccb725e01334e4b8e122c684ffc0e2e 2023-02-11 03:55:33,634 INFO [Listener at localhost/46205] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for bccb725e01334e4b8e122c684ffc0e2e 2023-02-11 03:55:33,635 INFO [Listener at localhost/46205] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in ea4735f297efbbfcc4cbc0bd3d6e42e0/family2 of ea4735f297efbbfcc4cbc0bd3d6e42e0 into bccb725e01334e4b8e122c684ffc0e2e(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-02-11 03:55:33,635 DEBUG [Listener at localhost/46205] regionserver.HRegion(2288): Compaction status journal for ea4735f297efbbfcc4cbc0bd3d6e42e0: 2023-02-11 03:55:33,640 DEBUG [Listener at localhost/46205] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-02-11 03:55:33,643 DEBUG [Listener at localhost/46205] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-02-11 03:55:33,647 DEBUG [Listener at localhost/46205] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-02-11 03:55:33,648 DEBUG [Listener at localhost/46205] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-02-11 03:55:33,650 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/ea4735f297efbbfcc4cbc0bd3d6e42e0/family2/7141c179434b4a0f816573a88da1caf7, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/ea4735f297efbbfcc4cbc0bd3d6e42e0/family2/045314dc19574fe9979fbedc8e14acf8, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/ea4735f297efbbfcc4cbc0bd3d6e42e0/family2/a77823d8bc254eb5b1130f536fa90187, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/ea4735f297efbbfcc4cbc0bd3d6e42e0/family2/5b41b5f362864186bb9f20144e3bc4c5, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/ea4735f297efbbfcc4cbc0bd3d6e42e0/family2/e7348b782602480c9a597e465ab9b946] to archive 2023-02-11 03:55:33,651 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-11 03:55:33,652 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/ea4735f297efbbfcc4cbc0bd3d6e42e0/family2/7141c179434b4a0f816573a88da1caf7 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testCompactionWithHFileLink/ea4735f297efbbfcc4cbc0bd3d6e42e0/family2/7141c179434b4a0f816573a88da1caf7 2023-02-11 03:55:33,653 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/ea4735f297efbbfcc4cbc0bd3d6e42e0/family2/045314dc19574fe9979fbedc8e14acf8 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testCompactionWithHFileLink/ea4735f297efbbfcc4cbc0bd3d6e42e0/family2/045314dc19574fe9979fbedc8e14acf8 2023-02-11 03:55:33,654 INFO [Listener at localhost/46205] compactions.PartitionedMobCompactor(142): is allFiles: false 2023-02-11 03:55:33,655 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/ea4735f297efbbfcc4cbc0bd3d6e42e0/family2/a77823d8bc254eb5b1130f536fa90187 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testCompactionWithHFileLink/ea4735f297efbbfcc4cbc0bd3d6e42e0/family2/a77823d8bc254eb5b1130f536fa90187 2023-02-11 03:55:33,656 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/ea4735f297efbbfcc4cbc0bd3d6e42e0/family2/5b41b5f362864186bb9f20144e3bc4c5 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testCompactionWithHFileLink/ea4735f297efbbfcc4cbc0bd3d6e42e0/family2/5b41b5f362864186bb9f20144e3bc4c5 2023-02-11 03:55:33,657 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/ea4735f297efbbfcc4cbc0bd3d6e42e0/family2/e7348b782602480c9a597e465ab9b946 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testCompactionWithHFileLink/ea4735f297efbbfcc4cbc0bd3d6e42e0/family2/e7348b782602480c9a597e465ab9b946 2023-02-11 03:55:33,686 INFO [Listener at localhost/46205] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 3 del files, 12 selected files, and 0 irrelevant files table 'testCompactionWithHFileLink' and column 'family1' 2023-02-11 03:55:33,689 INFO [Listener at localhost/46205] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e20230211bb7a69e13f50443eb5a9c83d8fe1e705_del 2023-02-11 03:55:33,692 INFO [Listener at localhost/46205] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b20230211a385f41cb4df40b08d76951be11968e4_del 2023-02-11 03:55:33,695 INFO [Listener at localhost/46205] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c2023021172810e71b3c3497c8cf0551a5ae35902_del 2023-02-11 03:55:33,695 INFO [Listener at localhost/46205] compactions.PartitionedMobCompactor(350): After merging, there are 3 del files. table='testCompactionWithHFileLink' column='family1' 2023-02-11 03:55:33,695 DEBUG [Listener at localhost/46205] zookeeper.ReadOnlyZKClient(139): Connect 0x18639285 to 127.0.0.1:53383 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-11 03:55:33,700 DEBUG [Listener at localhost/46205] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@79e2a56f, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-11 03:55:33,700 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c4ca4238a0b923820dcc509a6f75849b20230211 for table='testCompactionWithHFileLink' and column='family1' 2023-02-11 03:55:33,701 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c81e728d9d4c2f636f067f89cc14862c20230211 for table='testCompactionWithHFileLink' and column='family1' 2023-02-11 03:55:33,701 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20230211 for table='testCompactionWithHFileLink' and column='family1' 2023-02-11 03:55:34,163 INFO [MobFileCompactionChore-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b2023021125386759c08f4ff185b185945843fd79 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b2023021125386759c08f4ff185b185945843fd79 2023-02-11 03:55:34,163 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c4ca4238a0b923820dcc509a6f75849b20230211 table='testCompactionWithHFileLink' column='family1' 2023-02-11 03:55:34,176 DEBUG [MobFileCompactionChore-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-11 03:55:34,179 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:60476, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-11 03:55:34,185 DEBUG [MobFileCompactionChore-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-02-11 03:55:34,187 INFO [RS-EventLoopGroup-1-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:48660, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-02-11 03:55:34,198 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20230211/family1/e00ff403826943a5b2e72237b05de64d first=Optional[1\x00\x00\x00\x01] last=Optional[1\x00\x00\x00\x07] 2023-02-11 03:55:34,198 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testCompactionWithHFileLink,1,1676087711806.ea4735f297efbbfcc4cbc0bd3d6e42e0., hostname=jenkins-hbase4.apache.org,37011,1676087618428, seqNum=2 for row 1 with hfile group [{family1,hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20230211/family1/e00ff403826943a5b2e72237b05de64d}] 2023-02-11 03:55:34,202 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.HStore(763): Validating hfile at hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20230211/family1/e00ff403826943a5b2e72237b05de64d for inclusion in ea4735f297efbbfcc4cbc0bd3d6e42e0/family1 2023-02-11 03:55:34,204 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.HStore(775): HFile bounds: first=1\x00\x00\x00\x01 last=1\x00\x00\x00\x07 2023-02-11 03:55:34,204 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.HStore(777): Region bounds: first=1 last=2 2023-02-11 03:55:34,204 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.HRegion(2445): Flush status journal for ea4735f297efbbfcc4cbc0bd3d6e42e0: 2023-02-11 03:55:34,205 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20230211/family1/e00ff403826943a5b2e72237b05de64d to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/staging/jenkins__testCompactionWithHFileLink__efl1c4hq30q4a25cgu8di2rsavri27ubs2j2olm85mmpeku3qviv97qg13726bmk/family1/e00ff403826943a5b2e72237b05de64d 2023-02-11 03:55:34,206 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/staging/jenkins__testCompactionWithHFileLink__efl1c4hq30q4a25cgu8di2rsavri27ubs2j2olm85mmpeku3qviv97qg13726bmk/family1/e00ff403826943a5b2e72237b05de64d as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/ea4735f297efbbfcc4cbc0bd3d6e42e0/family1/a0d705d5ed3447a99303b7d9468cd95b_SeqId_29_ 2023-02-11 03:55:34,207 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.HStore(849): Loaded HFile hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/staging/jenkins__testCompactionWithHFileLink__efl1c4hq30q4a25cgu8di2rsavri27ubs2j2olm85mmpeku3qviv97qg13726bmk/family1/e00ff403826943a5b2e72237b05de64d into ea4735f297efbbfcc4cbc0bd3d6e42e0/family1 as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/ea4735f297efbbfcc4cbc0bd3d6e42e0/family1/a0d705d5ed3447a99303b7d9468cd95b_SeqId_29_ - updating store file list. 2023-02-11 03:55:34,209 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.HStore(882): Loaded HFile hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/ea4735f297efbbfcc4cbc0bd3d6e42e0/family1/a0d705d5ed3447a99303b7d9468cd95b_SeqId_29_ into ea4735f297efbbfcc4cbc0bd3d6e42e0/family1 2023-02-11 03:55:34,209 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.HStore(855): Successfully loaded hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/staging/jenkins__testCompactionWithHFileLink__efl1c4hq30q4a25cgu8di2rsavri27ubs2j2olm85mmpeku3qviv97qg13726bmk/family1/e00ff403826943a5b2e72237b05de64d into ea4735f297efbbfcc4cbc0bd3d6e42e0/family1 (new location: hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/ea4735f297efbbfcc4cbc0bd3d6e42e0/family1/a0d705d5ed3447a99303b7d9468cd95b_SeqId_29_) 2023-02-11 03:55:34,210 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/staging/jenkins__testCompactionWithHFileLink__efl1c4hq30q4a25cgu8di2rsavri27ubs2j2olm85mmpeku3qviv97qg13726bmk/family1/e00ff403826943a5b2e72237b05de64d 2023-02-11 03:55:34,212 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c4ca4238a0b923820dcc509a6f75849b20230211 table='testCompactionWithHFileLink' column='family1' 2023-02-11 03:55:34,212 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-11 03:55:34,213 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b2023021101c3e7c91909459eaa090b3ee73d4c03 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b2023021101c3e7c91909459eaa090b3ee73d4c03 2023-02-11 03:55:34,214 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b2023021143a646d95a904a22a34aeef0a1d76a19 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b2023021143a646d95a904a22a34aeef0a1d76a19 2023-02-11 03:55:34,215 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b2023021169ada67b5fa9484db7e2de93e45014c6 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b2023021169ada67b5fa9484db7e2de93e45014c6 2023-02-11 03:55:34,216 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230211bbcb00e25dbb444c96d85d72f4a0a09a to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230211bbcb00e25dbb444c96d85d72f4a0a09a 2023-02-11 03:55:34,216 INFO [MobFileCompactionChore-pool-0] mob.MobUtils(508): Table testCompactionWithHFileLink 4 expired mob files are deleted 2023-02-11 03:55:34,216 DEBUG [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20230211' 2023-02-11 03:55:34,216 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=c4ca4238a0b923820dcc509a6f75849b20230211 for table='testCompactionWithHFileLink' and column='family1' 2023-02-11 03:55:34,559 INFO [MobFileCompactionChore-pool-1] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230211fc38402a5e8e4cb4b9ced4bca92278bc to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230211fc38402a5e8e4cb4b9ced4bca92278bc 2023-02-11 03:55:34,559 INFO [MobFileCompactionChore-pool-2] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202302117c8611963e4f4e3cb549ecaa1eebeb1d to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202302117c8611963e4f4e3cb549ecaa1eebeb1d 2023-02-11 03:55:34,560 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20230211 table='testCompactionWithHFileLink' column='family1' 2023-02-11 03:55:34,560 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20230211 table='testCompactionWithHFileLink' column='family1' 2023-02-11 03:55:34,584 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20230211/family1/dc5afcf628ef461ca70cb205852a7896 first=Optional[0\x00\x00\x00\x01] last=Optional[0\x00\x00\x00\x07] 2023-02-11 03:55:34,584 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20230211/family1/1eecb912cda04d7b905224dd77bff561 first=Optional[2\x00\x00\x00\x01] last=Optional[2\x00\x00\x00\x07] 2023-02-11 03:55:34,586 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testCompactionWithHFileLink,,1676087711806.186d87f306418fb25d295d824c1272fe., hostname=jenkins-hbase4.apache.org,37011,1676087618428, seqNum=2 for row with hfile group [{family1,hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20230211/family1/dc5afcf628ef461ca70cb205852a7896}] 2023-02-11 03:55:34,586 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testCompactionWithHFileLink,2,1676087711806.621cd59809a16fd910172a9890d26263., hostname=jenkins-hbase4.apache.org,37011,1676087618428, seqNum=2 for row 2 with hfile group [{family1,hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20230211/family1/1eecb912cda04d7b905224dd77bff561}] 2023-02-11 03:55:34,588 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37011] regionserver.HStore(763): Validating hfile at hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20230211/family1/dc5afcf628ef461ca70cb205852a7896 for inclusion in 186d87f306418fb25d295d824c1272fe/family1 2023-02-11 03:55:34,588 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.HStore(763): Validating hfile at hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20230211/family1/1eecb912cda04d7b905224dd77bff561 for inclusion in 621cd59809a16fd910172a9890d26263/family1 2023-02-11 03:55:34,590 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.HStore(775): HFile bounds: first=2\x00\x00\x00\x01 last=2\x00\x00\x00\x07 2023-02-11 03:55:34,590 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37011] regionserver.HStore(775): HFile bounds: first=0\x00\x00\x00\x01 last=0\x00\x00\x00\x07 2023-02-11 03:55:34,590 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.HStore(777): Region bounds: first=2 last= 2023-02-11 03:55:34,590 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37011] regionserver.HStore(777): Region bounds: first= last=1 2023-02-11 03:55:34,590 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.HRegion(2445): Flush status journal for 621cd59809a16fd910172a9890d26263: 2023-02-11 03:55:34,590 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37011] regionserver.HRegion(2445): Flush status journal for 186d87f306418fb25d295d824c1272fe: 2023-02-11 03:55:34,591 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37011] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20230211/family1/dc5afcf628ef461ca70cb205852a7896 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/staging/jenkins__testCompactionWithHFileLink__n7g0vv978hj1f9cfvcd17khkkn7acfrmlrdu3usivpqat3dp0n8dstchd1evi56b/family1/dc5afcf628ef461ca70cb205852a7896 2023-02-11 03:55:34,592 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20230211/family1/1eecb912cda04d7b905224dd77bff561 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/staging/jenkins__testCompactionWithHFileLink__raon5v8c2ke28ko9quaq06qjli8f0acpl2gd0cqgpflruurueo001ha6mc4mcv7t/family1/1eecb912cda04d7b905224dd77bff561 2023-02-11 03:55:34,594 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/staging/jenkins__testCompactionWithHFileLink__raon5v8c2ke28ko9quaq06qjli8f0acpl2gd0cqgpflruurueo001ha6mc4mcv7t/family1/1eecb912cda04d7b905224dd77bff561 as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/621cd59809a16fd910172a9890d26263/family1/1982d73b09064ef3b63505c41585342c_SeqId_29_ 2023-02-11 03:55:34,594 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37011] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/staging/jenkins__testCompactionWithHFileLink__n7g0vv978hj1f9cfvcd17khkkn7acfrmlrdu3usivpqat3dp0n8dstchd1evi56b/family1/dc5afcf628ef461ca70cb205852a7896 as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/186d87f306418fb25d295d824c1272fe/family1/13b66e48c9e94749b690ffcbc521a3ef_SeqId_29_ 2023-02-11 03:55:34,595 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.HStore(849): Loaded HFile hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/staging/jenkins__testCompactionWithHFileLink__raon5v8c2ke28ko9quaq06qjli8f0acpl2gd0cqgpflruurueo001ha6mc4mcv7t/family1/1eecb912cda04d7b905224dd77bff561 into 621cd59809a16fd910172a9890d26263/family1 as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/621cd59809a16fd910172a9890d26263/family1/1982d73b09064ef3b63505c41585342c_SeqId_29_ - updating store file list. 2023-02-11 03:55:34,595 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37011] regionserver.HStore(849): Loaded HFile hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/staging/jenkins__testCompactionWithHFileLink__n7g0vv978hj1f9cfvcd17khkkn7acfrmlrdu3usivpqat3dp0n8dstchd1evi56b/family1/dc5afcf628ef461ca70cb205852a7896 into 186d87f306418fb25d295d824c1272fe/family1 as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/186d87f306418fb25d295d824c1272fe/family1/13b66e48c9e94749b690ffcbc521a3ef_SeqId_29_ - updating store file list. 2023-02-11 03:55:34,597 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.HStore(882): Loaded HFile hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/621cd59809a16fd910172a9890d26263/family1/1982d73b09064ef3b63505c41585342c_SeqId_29_ into 621cd59809a16fd910172a9890d26263/family1 2023-02-11 03:55:34,597 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.HStore(855): Successfully loaded hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/staging/jenkins__testCompactionWithHFileLink__raon5v8c2ke28ko9quaq06qjli8f0acpl2gd0cqgpflruurueo001ha6mc4mcv7t/family1/1eecb912cda04d7b905224dd77bff561 into 621cd59809a16fd910172a9890d26263/family1 (new location: hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/621cd59809a16fd910172a9890d26263/family1/1982d73b09064ef3b63505c41585342c_SeqId_29_) 2023-02-11 03:55:34,597 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37011] regionserver.HStore(882): Loaded HFile hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/186d87f306418fb25d295d824c1272fe/family1/13b66e48c9e94749b690ffcbc521a3ef_SeqId_29_ into 186d87f306418fb25d295d824c1272fe/family1 2023-02-11 03:55:34,598 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37011] regionserver.HStore(855): Successfully loaded hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/staging/jenkins__testCompactionWithHFileLink__n7g0vv978hj1f9cfvcd17khkkn7acfrmlrdu3usivpqat3dp0n8dstchd1evi56b/family1/dc5afcf628ef461ca70cb205852a7896 into 186d87f306418fb25d295d824c1272fe/family1 (new location: hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/186d87f306418fb25d295d824c1272fe/family1/13b66e48c9e94749b690ffcbc521a3ef_SeqId_29_) 2023-02-11 03:55:34,598 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/staging/jenkins__testCompactionWithHFileLink__raon5v8c2ke28ko9quaq06qjli8f0acpl2gd0cqgpflruurueo001ha6mc4mcv7t/family1/1eecb912cda04d7b905224dd77bff561 2023-02-11 03:55:34,598 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37011] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/staging/jenkins__testCompactionWithHFileLink__n7g0vv978hj1f9cfvcd17khkkn7acfrmlrdu3usivpqat3dp0n8dstchd1evi56b/family1/dc5afcf628ef461ca70cb205852a7896 2023-02-11 03:55:34,599 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20230211 table='testCompactionWithHFileLink' column='family1' 2023-02-11 03:55:34,600 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20230211 table='testCompactionWithHFileLink' column='family1' 2023-02-11 03:55:34,600 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-11 03:55:34,600 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(360): Archiving compacted files. 2023-02-11 03:55:34,601 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230211464ca8637b744ea09bffcf8850a4d558 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230211464ca8637b744ea09bffcf8850a4d558 2023-02-11 03:55:34,601 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230211101da7200e8b4cf48af3f9e6b626325b to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230211101da7200e8b4cf48af3f9e6b626325b 2023-02-11 03:55:34,602 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c2023021148e3d369a1484149bf86dd1e2169b387 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c2023021148e3d369a1484149bf86dd1e2169b387 2023-02-11 03:55:34,602 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230211e4a3d035f9844b869e82ec0923db93f1 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230211e4a3d035f9844b869e82ec0923db93f1 2023-02-11 03:55:34,603 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202302115097863fecd842c6a4c0b957f23811ff to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202302115097863fecd842c6a4c0b957f23811ff 2023-02-11 03:55:34,603 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230211e861d00e5a114d6b996d9db1875d1dc1 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230211e861d00e5a114d6b996d9db1875d1dc1 2023-02-11 03:55:34,604 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202302116f5da94e2cf44f26952495b89148e481 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202302116f5da94e2cf44f26952495b89148e481 2023-02-11 03:55:34,604 INFO [MobFileCompactionChore-pool-1] mob.MobUtils(508): Table testCompactionWithHFileLink 4 expired mob files are deleted 2023-02-11 03:55:34,604 DEBUG [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20230211' 2023-02-11 03:55:34,604 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230211efe856603053418f95a0d3b45cd7d38d to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230211efe856603053418f95a0d3b45cd7d38d 2023-02-11 03:55:34,604 INFO [MobFileCompactionChore-pool-2] mob.MobUtils(508): Table testCompactionWithHFileLink 4 expired mob files are deleted 2023-02-11 03:55:34,604 DEBUG [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20230211' 2023-02-11 03:55:34,605 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=c81e728d9d4c2f636f067f89cc14862c20230211 for table='testCompactionWithHFileLink' and column='family1' 2023-02-11 03:55:34,605 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20230211 for table='testCompactionWithHFileLink' and column='family1' 2023-02-11 03:55:34,605 INFO [Listener at localhost/46205] compactions.PartitionedMobCompactor(354): After compaction, there are 3 mob files. table='testCompactionWithHFileLink' column='family1' 2023-02-11 03:55:34,605 INFO [Listener at localhost/46205] compactions.PartitionedMobCompactor(364): After a mob compaction with all files selected, archiving the del files for table='testCompactionWithHFileLink' and column='family1' 2023-02-11 03:55:34,605 INFO [Listener at localhost/46205] compactions.PartitionedMobCompactor(367): [hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230211bb7a69e13f50443eb5a9c83d8fe1e705_del] 2023-02-11 03:55:34,605 DEBUG [Listener at localhost/46205] backup.HFileArchiver(360): Archiving compacted files. 2023-02-11 03:55:34,607 DEBUG [Listener at localhost/46205] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230211bb7a69e13f50443eb5a9c83d8fe1e705_del to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230211bb7a69e13f50443eb5a9c83d8fe1e705_del 2023-02-11 03:55:34,607 INFO [Listener at localhost/46205] mob.MobUtils(508): Table testCompactionWithHFileLink 1 expired mob files are deleted 2023-02-11 03:55:34,607 INFO [Listener at localhost/46205] compactions.PartitionedMobCompactor(367): [hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230211a385f41cb4df40b08d76951be11968e4_del] 2023-02-11 03:55:34,607 DEBUG [Listener at localhost/46205] backup.HFileArchiver(360): Archiving compacted files. 2023-02-11 03:55:34,608 DEBUG [Listener at localhost/46205] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230211a385f41cb4df40b08d76951be11968e4_del to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230211a385f41cb4df40b08d76951be11968e4_del 2023-02-11 03:55:34,608 INFO [Listener at localhost/46205] mob.MobUtils(508): Table testCompactionWithHFileLink 1 expired mob files are deleted 2023-02-11 03:55:34,608 INFO [Listener at localhost/46205] compactions.PartitionedMobCompactor(367): [hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c2023021172810e71b3c3497c8cf0551a5ae35902_del] 2023-02-11 03:55:34,609 DEBUG [Listener at localhost/46205] backup.HFileArchiver(360): Archiving compacted files. 2023-02-11 03:55:34,610 DEBUG [Listener at localhost/46205] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c2023021172810e71b3c3497c8cf0551a5ae35902_del to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c2023021172810e71b3c3497c8cf0551a5ae35902_del 2023-02-11 03:55:34,610 INFO [Listener at localhost/46205] mob.MobUtils(508): Table testCompactionWithHFileLink 1 expired mob files are deleted 2023-02-11 03:55:34,612 DEBUG [Listener at localhost/46205] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-02-11 03:55:34,614 DEBUG [Listener at localhost/46205] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-02-11 03:55:34,617 DEBUG [Listener at localhost/46205] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-02-11 03:55:34,617 DEBUG [Listener at localhost/46205] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-02-11 03:55:34,622 INFO [Listener at localhost/46205] client.HBaseAdmin$15(890): Started disable of testCompactionWithHFileLink 2023-02-11 03:55:34,622 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39921] master.HMaster$11(2405): Client=jenkins//172.31.14.131 disable testCompactionWithHFileLink 2023-02-11 03:55:34,623 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39921] procedure2.ProcedureExecutor(1029): Stored pid=88, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testCompactionWithHFileLink 2023-02-11 03:55:34,625 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39921] master.MasterRpcServices(1227): Checking to see if procedure is done pid=88 2023-02-11 03:55:34,625 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testCompactionWithHFileLink","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676087734625"}]},"ts":"1676087734625"} 2023-02-11 03:55:34,626 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=testCompactionWithHFileLink, state=DISABLING in hbase:meta 2023-02-11 03:55:34,628 INFO [PEWorker-3] procedure.DisableTableProcedure(293): Set testCompactionWithHFileLink to state=DISABLING 2023-02-11 03:55:34,628 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=89, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=186d87f306418fb25d295d824c1272fe, UNASSIGN}, {pid=90, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=ea4735f297efbbfcc4cbc0bd3d6e42e0, UNASSIGN}, {pid=91, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=621cd59809a16fd910172a9890d26263, UNASSIGN}] 2023-02-11 03:55:34,629 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=90, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=ea4735f297efbbfcc4cbc0bd3d6e42e0, UNASSIGN 2023-02-11 03:55:34,629 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=91, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=621cd59809a16fd910172a9890d26263, UNASSIGN 2023-02-11 03:55:34,630 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=89, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=186d87f306418fb25d295d824c1272fe, UNASSIGN 2023-02-11 03:55:34,630 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=90 updating hbase:meta row=ea4735f297efbbfcc4cbc0bd3d6e42e0, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:55:34,630 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=91 updating hbase:meta row=621cd59809a16fd910172a9890d26263, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:55:34,630 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=89 updating hbase:meta row=186d87f306418fb25d295d824c1272fe, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:55:34,630 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,1,1676087711806.ea4735f297efbbfcc4cbc0bd3d6e42e0.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1676087734630"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676087734630"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676087734630"}]},"ts":"1676087734630"} 2023-02-11 03:55:34,630 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,2,1676087711806.621cd59809a16fd910172a9890d26263.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1676087734630"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676087734630"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676087734630"}]},"ts":"1676087734630"} 2023-02-11 03:55:34,630 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,,1676087711806.186d87f306418fb25d295d824c1272fe.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1676087734630"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676087734630"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676087734630"}]},"ts":"1676087734630"} 2023-02-11 03:55:34,631 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=92, ppid=90, state=RUNNABLE; CloseRegionProcedure ea4735f297efbbfcc4cbc0bd3d6e42e0, server=jenkins-hbase4.apache.org,37011,1676087618428}] 2023-02-11 03:55:34,632 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=93, ppid=91, state=RUNNABLE; CloseRegionProcedure 621cd59809a16fd910172a9890d26263, server=jenkins-hbase4.apache.org,37011,1676087618428}] 2023-02-11 03:55:34,633 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=94, ppid=89, state=RUNNABLE; CloseRegionProcedure 186d87f306418fb25d295d824c1272fe, server=jenkins-hbase4.apache.org,37011,1676087618428}] 2023-02-11 03:55:34,726 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39921] master.MasterRpcServices(1227): Checking to see if procedure is done pid=88 2023-02-11 03:55:34,783 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 621cd59809a16fd910172a9890d26263 2023-02-11 03:55:34,784 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 621cd59809a16fd910172a9890d26263, disabling compactions & flushes 2023-02-11 03:55:34,784 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testCompactionWithHFileLink,2,1676087711806.621cd59809a16fd910172a9890d26263. 2023-02-11 03:55:34,784 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testCompactionWithHFileLink,2,1676087711806.621cd59809a16fd910172a9890d26263. 2023-02-11 03:55:34,784 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testCompactionWithHFileLink,2,1676087711806.621cd59809a16fd910172a9890d26263. after waiting 0 ms 2023-02-11 03:55:34,784 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testCompactionWithHFileLink,2,1676087711806.621cd59809a16fd910172a9890d26263. 2023-02-11 03:55:34,788 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/621cd59809a16fd910172a9890d26263/recovered.edits/31.seqid, newMaxSeqId=31, maxSeqId=1 2023-02-11 03:55:34,789 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testCompactionWithHFileLink,2,1676087711806.621cd59809a16fd910172a9890d26263. 2023-02-11 03:55:34,789 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 621cd59809a16fd910172a9890d26263: Waiting for close lock at 1676087734784Running coprocessor pre-close hooks at 1676087734784Disabling compacts and flushes for region at 1676087734784Disabling writes for close at 1676087734784Writing region close event to WAL at 1676087734785 (+1 ms)Running coprocessor post-close hooks at 1676087734788 (+3 ms)Closed at 1676087734789 (+1 ms) 2023-02-11 03:55:34,790 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 621cd59809a16fd910172a9890d26263 2023-02-11 03:55:34,791 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close ea4735f297efbbfcc4cbc0bd3d6e42e0 2023-02-11 03:55:34,791 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing ea4735f297efbbfcc4cbc0bd3d6e42e0, disabling compactions & flushes 2023-02-11 03:55:34,791 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testCompactionWithHFileLink,1,1676087711806.ea4735f297efbbfcc4cbc0bd3d6e42e0. 2023-02-11 03:55:34,791 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testCompactionWithHFileLink,1,1676087711806.ea4735f297efbbfcc4cbc0bd3d6e42e0. 2023-02-11 03:55:34,791 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testCompactionWithHFileLink,1,1676087711806.ea4735f297efbbfcc4cbc0bd3d6e42e0. after waiting 0 ms 2023-02-11 03:55:34,791 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testCompactionWithHFileLink,1,1676087711806.ea4735f297efbbfcc4cbc0bd3d6e42e0. 2023-02-11 03:55:34,791 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=91 updating hbase:meta row=621cd59809a16fd910172a9890d26263, regionState=CLOSED 2023-02-11 03:55:34,791 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,2,1676087711806.621cd59809a16fd910172a9890d26263.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1676087734791"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676087734791"}]},"ts":"1676087734791"} 2023-02-11 03:55:34,794 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=93, resume processing ppid=91 2023-02-11 03:55:34,794 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=93, ppid=91, state=SUCCESS; CloseRegionProcedure 621cd59809a16fd910172a9890d26263, server=jenkins-hbase4.apache.org,37011,1676087618428 in 160 msec 2023-02-11 03:55:34,794 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/ea4735f297efbbfcc4cbc0bd3d6e42e0/recovered.edits/31.seqid, newMaxSeqId=31, maxSeqId=1 2023-02-11 03:55:34,795 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=91, ppid=88, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=621cd59809a16fd910172a9890d26263, UNASSIGN in 166 msec 2023-02-11 03:55:34,796 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testCompactionWithHFileLink,1,1676087711806.ea4735f297efbbfcc4cbc0bd3d6e42e0. 2023-02-11 03:55:34,796 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for ea4735f297efbbfcc4cbc0bd3d6e42e0: Waiting for close lock at 1676087734791Running coprocessor pre-close hooks at 1676087734791Disabling compacts and flushes for region at 1676087734791Disabling writes for close at 1676087734791Writing region close event to WAL at 1676087734792 (+1 ms)Running coprocessor post-close hooks at 1676087734795 (+3 ms)Closed at 1676087734796 (+1 ms) 2023-02-11 03:55:34,797 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed ea4735f297efbbfcc4cbc0bd3d6e42e0 2023-02-11 03:55:34,797 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 186d87f306418fb25d295d824c1272fe 2023-02-11 03:55:34,797 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 186d87f306418fb25d295d824c1272fe, disabling compactions & flushes 2023-02-11 03:55:34,797 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testCompactionWithHFileLink,,1676087711806.186d87f306418fb25d295d824c1272fe. 2023-02-11 03:55:34,797 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testCompactionWithHFileLink,,1676087711806.186d87f306418fb25d295d824c1272fe. 2023-02-11 03:55:34,797 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testCompactionWithHFileLink,,1676087711806.186d87f306418fb25d295d824c1272fe. after waiting 0 ms 2023-02-11 03:55:34,797 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testCompactionWithHFileLink,,1676087711806.186d87f306418fb25d295d824c1272fe. 2023-02-11 03:55:34,797 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=90 updating hbase:meta row=ea4735f297efbbfcc4cbc0bd3d6e42e0, regionState=CLOSED 2023-02-11 03:55:34,797 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,1,1676087711806.ea4735f297efbbfcc4cbc0bd3d6e42e0.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1676087734797"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676087734797"}]},"ts":"1676087734797"} 2023-02-11 03:55:34,799 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=92, resume processing ppid=90 2023-02-11 03:55:34,799 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=92, ppid=90, state=SUCCESS; CloseRegionProcedure ea4735f297efbbfcc4cbc0bd3d6e42e0, server=jenkins-hbase4.apache.org,37011,1676087618428 in 167 msec 2023-02-11 03:55:34,800 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=90, ppid=88, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=ea4735f297efbbfcc4cbc0bd3d6e42e0, UNASSIGN in 171 msec 2023-02-11 03:55:34,800 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/186d87f306418fb25d295d824c1272fe/recovered.edits/31.seqid, newMaxSeqId=31, maxSeqId=1 2023-02-11 03:55:34,801 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testCompactionWithHFileLink,,1676087711806.186d87f306418fb25d295d824c1272fe. 2023-02-11 03:55:34,801 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 186d87f306418fb25d295d824c1272fe: Waiting for close lock at 1676087734797Running coprocessor pre-close hooks at 1676087734797Disabling compacts and flushes for region at 1676087734797Disabling writes for close at 1676087734797Writing region close event to WAL at 1676087734798 (+1 ms)Running coprocessor post-close hooks at 1676087734801 (+3 ms)Closed at 1676087734801 2023-02-11 03:55:34,802 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 186d87f306418fb25d295d824c1272fe 2023-02-11 03:55:34,803 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=89 updating hbase:meta row=186d87f306418fb25d295d824c1272fe, regionState=CLOSED 2023-02-11 03:55:34,803 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,,1676087711806.186d87f306418fb25d295d824c1272fe.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1676087734803"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676087734803"}]},"ts":"1676087734803"} 2023-02-11 03:55:34,805 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=94, resume processing ppid=89 2023-02-11 03:55:34,805 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=94, ppid=89, state=SUCCESS; CloseRegionProcedure 186d87f306418fb25d295d824c1272fe, server=jenkins-hbase4.apache.org,37011,1676087618428 in 171 msec 2023-02-11 03:55:34,806 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=89, resume processing ppid=88 2023-02-11 03:55:34,806 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=89, ppid=88, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=186d87f306418fb25d295d824c1272fe, UNASSIGN in 177 msec 2023-02-11 03:55:34,806 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testCompactionWithHFileLink","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676087734806"}]},"ts":"1676087734806"} 2023-02-11 03:55:34,807 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testCompactionWithHFileLink, state=DISABLED in hbase:meta 2023-02-11 03:55:34,808 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testCompactionWithHFileLink to state=DISABLED 2023-02-11 03:55:34,810 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=88, state=SUCCESS; DisableTableProcedure table=testCompactionWithHFileLink in 187 msec 2023-02-11 03:55:34,927 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39921] master.MasterRpcServices(1227): Checking to see if procedure is done pid=88 2023-02-11 03:55:34,927 INFO [Listener at localhost/46205] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testCompactionWithHFileLink, procId: 88 completed 2023-02-11 03:55:34,937 INFO [Listener at localhost/46205] client.HBaseAdmin(2586): Taking restore-failsafe snapshot: hbase-failsafe-snaptb-1676087720079-1676087734937 2023-02-11 03:55:34,938 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39921] master.MasterRpcServices(1580): Client=jenkins//172.31.14.131 snapshot request for:{ ss=hbase-failsafe-snaptb-1676087720079-1676087734937 table=testCompactionWithHFileLink type=FLUSH ttl=0 } 2023-02-11 03:55:34,938 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39921] snapshot.SnapshotDescriptionUtils(316): Creation time not specified, setting to:1676087734938 (current time:1676087734938). 2023-02-11 03:55:34,938 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39921] snapshot.SnapshotDescriptionUtils(332): Snapshot current TTL value: 0 resetting it to default value: 0 2023-02-11 03:55:34,938 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39921] zookeeper.ReadOnlyZKClient(139): Connect 0x6ced99e7 to 127.0.0.1:53383 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-11 03:55:34,943 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39921] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@4ce7a19, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-11 03:55:34,944 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39921] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-11 03:55:34,946 INFO [RS-EventLoopGroup-3-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:60482, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-11 03:55:34,947 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39921] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x6ced99e7 to 127.0.0.1:53383 2023-02-11 03:55:34,947 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39921] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-11 03:55:34,947 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39921] snapshot.SnapshotManager(601): No existing snapshot, attempting snapshot... 2023-02-11 03:55:34,947 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39921] snapshot.SnapshotManager(659): Table is disabled, running snapshot entirely on master for { ss=hbase-failsafe-snaptb-1676087720079-1676087734937 table=testCompactionWithHFileLink type=FLUSH ttl=0 } 2023-02-11 03:55:34,949 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39921] procedure2.ProcedureExecutor(1029): Stored pid=95, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE 2023-02-11 03:55:34,950 DEBUG [PEWorker-5] locking.LockProcedure(309): LOCKED pid=95, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE 2023-02-11 03:55:34,950 INFO [PEWorker-5] procedure2.TimeoutExecutorThread(81): ADDED pid=95, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE; timeout=600000, timestamp=1676088334950 2023-02-11 03:55:34,950 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39921] snapshot.SnapshotManager(664): Started snapshot: { ss=hbase-failsafe-snaptb-1676087720079-1676087734937 table=testCompactionWithHFileLink type=FLUSH ttl=0 } 2023-02-11 03:55:34,950 INFO [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.TakeSnapshotHandler(174): Running DISABLED table snapshot hbase-failsafe-snaptb-1676087720079-1676087734937 C_M_SNAPSHOT_TABLE on table testCompactionWithHFileLink 2023-02-11 03:55:34,951 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2418): Waiting a max of 300000 ms for snapshot '{ ss=hbase-failsafe-snaptb-1676087720079-1676087734937 table=testCompactionWithHFileLink type=FLUSH ttl=0 }'' to complete. (max 300000 ms per retry) 2023-02-11 03:55:34,951 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2428): (#1) Sleeping: 100ms while waiting for snapshot completion. 2023-02-11 03:55:35,051 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-02-11 03:55:35,052 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39921] master.MasterRpcServices(1209): Checking to see if snapshot from request:{ ss=hbase-failsafe-snaptb-1676087720079-1676087734937 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-02-11 03:55:35,052 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39921] snapshot.SnapshotManager(404): Snapshoting '{ ss=hbase-failsafe-snaptb-1676087720079-1676087734937 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' is still in progress! 2023-02-11 03:55:35,053 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2428): (#2) Sleeping: 200ms while waiting for snapshot completion. 2023-02-11 03:55:35,253 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-02-11 03:55:35,254 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39921] master.MasterRpcServices(1209): Checking to see if snapshot from request:{ ss=hbase-failsafe-snaptb-1676087720079-1676087734937 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-02-11 03:55:35,254 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39921] snapshot.SnapshotManager(404): Snapshoting '{ ss=hbase-failsafe-snaptb-1676087720079-1676087734937 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' is still in progress! 2023-02-11 03:55:35,254 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2428): (#3) Sleeping: 300ms while waiting for snapshot completion. 2023-02-11 03:55:35,358 INFO [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.DisabledTableSnapshotHandler(97): Starting to write region info and WALs for regions for offline snapshot:{ ss=hbase-failsafe-snaptb-1676087720079-1676087734937 table=testCompactionWithHFileLink type=DISABLED ttl=0 } 2023-02-11 03:55:35,359 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(297): Storing region-info for snapshot. 2023-02-11 03:55:35,359 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(297): Storing region-info for snapshot. 2023-02-11 03:55:35,359 DEBUG [DisabledTableSnapshot-pool-2] snapshot.SnapshotManifest(297): Storing region-info for snapshot. 2023-02-11 03:55:35,359 DEBUG [DisabledTableSnapshot-pool-2] snapshot.SnapshotManifest(302): Creating references for hfiles 2023-02-11 03:55:35,359 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(302): Creating references for hfiles 2023-02-11 03:55:35,359 DEBUG [DisabledTableSnapshot-pool-3] snapshot.SnapshotManifest(297): Storing region-info for snapshot. 2023-02-11 03:55:35,359 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(302): Creating references for hfiles 2023-02-11 03:55:35,360 DEBUG [DisabledTableSnapshot-pool-3] snapshot.SnapshotManifest(302): Creating references for hfiles 2023-02-11 03:55:35,362 DEBUG [DisabledTableSnapshot-pool-2] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/186d87f306418fb25d295d824c1272fe/family1/13b66e48c9e94749b690ffcbc521a3ef_SeqId_29_, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/186d87f306418fb25d295d824c1272fe/family1/33bb864dff41474f9562086b84b9018e] hfiles 2023-02-11 03:55:35,362 DEBUG [DisabledTableSnapshot-pool-2] snapshot.SnapshotManifest(363): Adding reference for hfile (1/2): hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/186d87f306418fb25d295d824c1272fe/family1/13b66e48c9e94749b690ffcbc521a3ef_SeqId_29_ 2023-02-11 03:55:35,362 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/621cd59809a16fd910172a9890d26263/family1/1982d73b09064ef3b63505c41585342c_SeqId_29_, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/621cd59809a16fd910172a9890d26263/family1/6b911754855f4041855ad51000f60d9e] hfiles 2023-02-11 03:55:35,362 DEBUG [DisabledTableSnapshot-pool-2] snapshot.SnapshotManifest(363): Adding reference for hfile (2/2): hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/186d87f306418fb25d295d824c1272fe/family1/33bb864dff41474f9562086b84b9018e 2023-02-11 03:55:35,362 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(363): Adding reference for hfile (1/2): hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/621cd59809a16fd910172a9890d26263/family1/1982d73b09064ef3b63505c41585342c_SeqId_29_ 2023-02-11 03:55:35,363 DEBUG [DisabledTableSnapshot-pool-3] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/ea4735f297efbbfcc4cbc0bd3d6e42e0/family1/553adfdab49e4996b9384047185ed50a, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/ea4735f297efbbfcc4cbc0bd3d6e42e0/family1/a0d705d5ed3447a99303b7d9468cd95b_SeqId_29_] hfiles 2023-02-11 03:55:35,362 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b2023021125386759c08f4ff185b185945843fd79, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230211fc38402a5e8e4cb4b9ced4bca92278bc, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202302117c8611963e4f4e3cb549ecaa1eebeb1d] hfiles 2023-02-11 03:55:35,363 DEBUG [DisabledTableSnapshot-pool-3] snapshot.SnapshotManifest(363): Adding reference for hfile (1/2): hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/ea4735f297efbbfcc4cbc0bd3d6e42e0/family1/553adfdab49e4996b9384047185ed50a 2023-02-11 03:55:35,363 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(363): Adding reference for hfile (2/2): hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/621cd59809a16fd910172a9890d26263/family1/6b911754855f4041855ad51000f60d9e 2023-02-11 03:55:35,363 DEBUG [DisabledTableSnapshot-pool-3] snapshot.SnapshotManifest(363): Adding reference for hfile (2/2): hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/ea4735f297efbbfcc4cbc0bd3d6e42e0/family1/a0d705d5ed3447a99303b7d9468cd95b_SeqId_29_ 2023-02-11 03:55:35,363 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (1/3): hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b2023021125386759c08f4ff185b185945843fd79 2023-02-11 03:55:35,363 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (2/3): hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230211fc38402a5e8e4cb4b9ced4bca92278bc 2023-02-11 03:55:35,363 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (3/3): hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202302117c8611963e4f4e3cb549ecaa1eebeb1d 2023-02-11 03:55:35,363 DEBUG [DisabledTableSnapshot-pool-2] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/186d87f306418fb25d295d824c1272fe/family2/6046f3d6ed634b868e201512cc6356ce] hfiles 2023-02-11 03:55:35,363 DEBUG [DisabledTableSnapshot-pool-2] snapshot.SnapshotManifest(363): Adding reference for hfile (1/1): hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/186d87f306418fb25d295d824c1272fe/family2/6046f3d6ed634b868e201512cc6356ce 2023-02-11 03:55:35,364 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/621cd59809a16fd910172a9890d26263/family2/8e6e96733be0435d8cd67d27596328ca] hfiles 2023-02-11 03:55:35,364 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(363): Adding reference for hfile (1/1): hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/621cd59809a16fd910172a9890d26263/family2/8e6e96733be0435d8cd67d27596328ca 2023-02-11 03:55:35,364 DEBUG [DisabledTableSnapshot-pool-3] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/ea4735f297efbbfcc4cbc0bd3d6e42e0/family2/bccb725e01334e4b8e122c684ffc0e2e] hfiles 2023-02-11 03:55:35,364 DEBUG [DisabledTableSnapshot-pool-3] snapshot.SnapshotManifest(363): Adding reference for hfile (1/1): hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/ea4735f297efbbfcc4cbc0bd3d6e42e0/family2/bccb725e01334e4b8e122c684ffc0e2e 2023-02-11 03:55:35,369 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230211211c0620d3924f14baf3c085690d9d43, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202302114071a4681ba54ea288ea82635207f6ae, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b2023021165313005ab884230948c8003bec6d4ef, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202302116fc53053458b4569a977face277d8136, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230211a0ddb0b1c2124b71a00870343e6a77d3_del, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c202302110d01c3b1b7ff473d89c5b36246154d46, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c202302117e5060c420684526926d62b7b980770c, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230211cc6d231f112e474eb4759acadde745f1, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230211d2516239b51e4745bb93c4eebc72cef9_del, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230211e4d7df1436cd4bb0a383daae94da9f0f, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e202302110f3eef6bbc07470d998a60ead6aad577, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e2023021188469be17b5f4d66bfeeb655b336e842, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230211ace870e2042e406dbd52145d1caaaf37, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230211bf6b892b644c4f1fab3514747a0eaf2a_del, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230211cb55ca343ae8440a83b4999e2a1b2131] hfiles 2023-02-11 03:55:35,369 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (1/15): hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230211211c0620d3924f14baf3c085690d9d43 2023-02-11 03:55:35,369 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (2/15): hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202302114071a4681ba54ea288ea82635207f6ae 2023-02-11 03:55:35,369 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (3/15): hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b2023021165313005ab884230948c8003bec6d4ef 2023-02-11 03:55:35,369 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (4/15): hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202302116fc53053458b4569a977face277d8136 2023-02-11 03:55:35,369 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (5/15): hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230211a0ddb0b1c2124b71a00870343e6a77d3_del 2023-02-11 03:55:35,369 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (6/15): hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c202302110d01c3b1b7ff473d89c5b36246154d46 2023-02-11 03:55:35,370 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (7/15): hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c202302117e5060c420684526926d62b7b980770c 2023-02-11 03:55:35,370 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (8/15): hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230211cc6d231f112e474eb4759acadde745f1 2023-02-11 03:55:35,370 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (9/15): hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230211d2516239b51e4745bb93c4eebc72cef9_del 2023-02-11 03:55:35,370 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (10/15): hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230211e4d7df1436cd4bb0a383daae94da9f0f 2023-02-11 03:55:35,370 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (11/15): hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e202302110f3eef6bbc07470d998a60ead6aad577 2023-02-11 03:55:35,370 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (12/15): hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e2023021188469be17b5f4d66bfeeb655b336e842 2023-02-11 03:55:35,370 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (13/15): hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230211ace870e2042e406dbd52145d1caaaf37 2023-02-11 03:55:35,370 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (14/15): hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230211bf6b892b644c4f1fab3514747a0eaf2a_del 2023-02-11 03:55:35,370 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (15/15): hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230211cb55ca343ae8440a83b4999e2a1b2131 2023-02-11 03:55:35,554 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-02-11 03:55:35,555 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39921] master.MasterRpcServices(1209): Checking to see if snapshot from request:{ ss=hbase-failsafe-snaptb-1676087720079-1676087734937 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-02-11 03:55:35,555 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39921] snapshot.SnapshotManager(404): Snapshoting '{ ss=hbase-failsafe-snaptb-1676087720079-1676087734937 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' is still in progress! 2023-02-11 03:55:35,555 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2428): (#4) Sleeping: 500ms while waiting for snapshot completion. 2023-02-11 03:55:35,786 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.DisabledTableSnapshotHandler(120): Marking snapshot{ ss=hbase-failsafe-snaptb-1676087720079-1676087734937 table=testCompactionWithHFileLink type=DISABLED ttl=0 } as finished. 2023-02-11 03:55:35,786 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(484): Convert to Single Snapshot Manifest for hbase-failsafe-snaptb-1676087720079-1676087734937 2023-02-11 03:55:35,786 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifestV1(126): No regions under directory:hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/.hbase-snapshot/.tmp/hbase-failsafe-snaptb-1676087720079-1676087734937 2023-02-11 03:55:36,055 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-02-11 03:55:36,056 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39921] master.MasterRpcServices(1209): Checking to see if snapshot from request:{ ss=hbase-failsafe-snaptb-1676087720079-1676087734937 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-02-11 03:55:36,056 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39921] snapshot.SnapshotManager(404): Snapshoting '{ ss=hbase-failsafe-snaptb-1676087720079-1676087734937 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' is still in progress! 2023-02-11 03:55:36,056 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2428): (#5) Sleeping: 1000ms while waiting for snapshot completion. 2023-02-11 03:55:36,206 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotDescriptionUtils(404): Sentinel is done, just moving the snapshot from hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/.hbase-snapshot/.tmp/hbase-failsafe-snaptb-1676087720079-1676087734937 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/.hbase-snapshot/hbase-failsafe-snaptb-1676087720079-1676087734937 2023-02-11 03:55:37,020 INFO [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.TakeSnapshotHandler(229): Snapshot hbase-failsafe-snaptb-1676087720079-1676087734937 of table testCompactionWithHFileLink completed 2023-02-11 03:55:37,020 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.TakeSnapshotHandler(246): Launching cleanup of working dir:hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/.hbase-snapshot/.tmp/hbase-failsafe-snaptb-1676087720079-1676087734937 2023-02-11 03:55:37,021 ERROR [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.TakeSnapshotHandler(251): Couldn't delete snapshot working directory:hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/.hbase-snapshot/.tmp/hbase-failsafe-snaptb-1676087720079-1676087734937 2023-02-11 03:55:37,021 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.TakeSnapshotHandler(257): Table snapshot journal : Running DISABLED table snapshot hbase-failsafe-snaptb-1676087720079-1676087734937 C_M_SNAPSHOT_TABLE on table testCompactionWithHFileLink at 1676087734951Starting to write region info and WALs for regions for offline snapshot:{ ss=hbase-failsafe-snaptb-1676087720079-1676087734937 table=testCompactionWithHFileLink type=DISABLED ttl=0 } at 1676087735358 (+407 ms)Consolidate snapshot: hbase-failsafe-snaptb-1676087720079-1676087734937 at 1676087735786 (+428 ms)Loading Region manifests for hbase-failsafe-snaptb-1676087720079-1676087734937 at 1676087735786Writing data manifest for hbase-failsafe-snaptb-1676087720079-1676087734937 at 1676087735788 (+2 ms)Verifying snapshot: hbase-failsafe-snaptb-1676087720079-1676087734937 at 1676087736194 (+406 ms)Snapshot hbase-failsafe-snaptb-1676087720079-1676087734937 of table testCompactionWithHFileLink completed at 1676087737020 (+826 ms) 2023-02-11 03:55:37,022 DEBUG [PEWorker-3] locking.LockProcedure(242): UNLOCKED pid=95, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE 2023-02-11 03:55:37,023 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=95, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE in 2.0730 sec 2023-02-11 03:55:37,056 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-02-11 03:55:37,057 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39921] master.MasterRpcServices(1209): Checking to see if snapshot from request:{ ss=hbase-failsafe-snaptb-1676087720079-1676087734937 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-02-11 03:55:37,057 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39921] snapshot.SnapshotManager(401): Snapshot '{ ss=hbase-failsafe-snaptb-1676087720079-1676087734937 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' has completed, notifying client. 2023-02-11 03:55:37,097 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39921] procedure2.ProcedureExecutor(1029): Stored pid=96, state=RUNNABLE:RESTORE_SNAPSHOT_PRE_OPERATION; RestoreSnapshotProcedure (table=testCompactionWithHFileLink snapshot=name: "snaptb-1676087720079" table: "testCompactionWithHFileLink" creation_time: 1676087720089 type: FLUSH version: 2 ttl: 0 ) 2023-02-11 03:55:37,097 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39921] snapshot.SnapshotManager(880): Restore snapshot=snaptb-1676087720079 as table=testCompactionWithHFileLink 2023-02-11 03:55:37,100 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39921] master.MasterRpcServices(1227): Checking to see if procedure is done pid=96 2023-02-11 03:55:37,201 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39921] master.MasterRpcServices(1227): Checking to see if procedure is done pid=96 2023-02-11 03:55:37,402 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39921] master.MasterRpcServices(1227): Checking to see if procedure is done pid=96 2023-02-11 03:55:37,507 DEBUG [PEWorker-2] util.FSTableDescriptors(570): Wrote into hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/.tabledesc/.tableinfo.0000000002 2023-02-11 03:55:37,508 DEBUG [PEWorker-2] util.FSTableDescriptors(519): Deleted hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/.tabledesc/.tableinfo.0000000001 2023-02-11 03:55:37,508 INFO [PEWorker-2] util.FSTableDescriptors(293): Updated tableinfo=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/.tabledesc/.tableinfo.0000000002 2023-02-11 03:55:37,509 INFO [PEWorker-2] procedure.RestoreSnapshotProcedure(374): Starting restore snapshot={ ss=snaptb-1676087720079 table=testCompactionWithHFileLink type=FLUSH ttl=0 } 2023-02-11 03:55:37,513 INFO [PEWorker-2] snapshot.RestoreSnapshotHelper(177): starting restore table regions using snapshot=name: "snaptb-1676087720079" table: "testCompactionWithHFileLink" creation_time: 1676087720089 type: FLUSH version: 2 ttl: 0 2023-02-11 03:55:37,513 DEBUG [PEWorker-2] snapshot.RestoreSnapshotHelper(785): get table regions: hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink 2023-02-11 03:55:37,518 DEBUG [PEWorker-2] snapshot.RestoreSnapshotHelper(797): found 3 regions for table=testCompactionWithHFileLink 2023-02-11 03:55:37,521 DEBUG [RestoreSnapshot-pool-0] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202302117c8611963e4f4e3cb549ecaa1eebeb1d to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202302117c8611963e4f4e3cb549ecaa1eebeb1d 2023-02-11 03:55:37,522 DEBUG [RestoreSnapshot-pool-0] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b2023021125386759c08f4ff185b185945843fd79 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b2023021125386759c08f4ff185b185945843fd79 2023-02-11 03:55:37,523 DEBUG [RestoreSnapshot-pool-0] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230211fc38402a5e8e4cb4b9ced4bca92278bc to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230211fc38402a5e8e4cb4b9ced4bca92278bc 2023-02-11 03:55:37,524 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c4ca4238a0b923820dcc509a6f75849b2023021101c3e7c91909459eaa090b3ee73d4c03 of snapshot=snaptb-1676087720079 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-02-11 03:55:37,527 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c4ca4238a0b923820dcc509a6f75849b2023021143a646d95a904a22a34aeef0a1d76a19 of snapshot=snaptb-1676087720079 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-02-11 03:55:37,530 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c4ca4238a0b923820dcc509a6f75849b2023021169ada67b5fa9484db7e2de93e45014c6 of snapshot=snaptb-1676087720079 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-02-11 03:55:37,532 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c4ca4238a0b923820dcc509a6f75849b20230211bbcb00e25dbb444c96d85d72f4a0a09a of snapshot=snaptb-1676087720079 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-02-11 03:55:37,535 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c81e728d9d4c2f636f067f89cc14862c20230211464ca8637b744ea09bffcf8850a4d558 of snapshot=snaptb-1676087720079 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-02-11 03:55:37,538 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c81e728d9d4c2f636f067f89cc14862c2023021148e3d369a1484149bf86dd1e2169b387 of snapshot=snaptb-1676087720079 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-02-11 03:55:37,541 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c81e728d9d4c2f636f067f89cc14862c202302115097863fecd842c6a4c0b957f23811ff of snapshot=snaptb-1676087720079 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-02-11 03:55:37,543 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c81e728d9d4c2f636f067f89cc14862c202302116f5da94e2cf44f26952495b89148e481 of snapshot=snaptb-1676087720079 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-02-11 03:55:37,546 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink d41d8cd98f00b204e9800998ecf8427e20230211101da7200e8b4cf48af3f9e6b626325b of snapshot=snaptb-1676087720079 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-02-11 03:55:37,549 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink d41d8cd98f00b204e9800998ecf8427e20230211e4a3d035f9844b869e82ec0923db93f1 of snapshot=snaptb-1676087720079 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-02-11 03:55:37,551 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink d41d8cd98f00b204e9800998ecf8427e20230211e861d00e5a114d6b996d9db1875d1dc1 of snapshot=snaptb-1676087720079 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-02-11 03:55:37,554 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink d41d8cd98f00b204e9800998ecf8427e20230211efe856603053418f95a0d3b45cd7d38d of snapshot=snaptb-1676087720079 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-02-11 03:55:37,559 DEBUG [RestoreSnapshot-pool-0] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230211a0ddb0b1c2124b71a00870343e6a77d3_del to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230211a0ddb0b1c2124b71a00870343e6a77d3_del 2023-02-11 03:55:37,560 DEBUG [RestoreSnapshot-pool-0] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230211bf6b892b644c4f1fab3514747a0eaf2a_del to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230211bf6b892b644c4f1fab3514747a0eaf2a_del 2023-02-11 03:55:37,561 DEBUG [RestoreSnapshot-pool-0] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230211d2516239b51e4745bb93c4eebc72cef9_del to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230211d2516239b51e4745bb93c4eebc72cef9_del 2023-02-11 03:55:37,561 INFO [PEWorker-2] snapshot.RestoreSnapshotHelper(223): region to restore: 186d87f306418fb25d295d824c1272fe 2023-02-11 03:55:37,561 INFO [PEWorker-2] snapshot.RestoreSnapshotHelper(223): region to restore: 621cd59809a16fd910172a9890d26263 2023-02-11 03:55:37,561 INFO [PEWorker-2] snapshot.RestoreSnapshotHelper(223): region to restore: ea4735f297efbbfcc4cbc0bd3d6e42e0 2023-02-11 03:55:37,564 DEBUG [RestoreSnapshot-pool-2] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/621cd59809a16fd910172a9890d26263/family1/6b911754855f4041855ad51000f60d9e to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testCompactionWithHFileLink/621cd59809a16fd910172a9890d26263/family1/6b911754855f4041855ad51000f60d9e 2023-02-11 03:55:37,564 DEBUG [RestoreSnapshot-pool-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/ea4735f297efbbfcc4cbc0bd3d6e42e0/family1/553adfdab49e4996b9384047185ed50a to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testCompactionWithHFileLink/ea4735f297efbbfcc4cbc0bd3d6e42e0/family1/553adfdab49e4996b9384047185ed50a 2023-02-11 03:55:37,565 DEBUG [RestoreSnapshot-pool-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/186d87f306418fb25d295d824c1272fe/family1/13b66e48c9e94749b690ffcbc521a3ef_SeqId_29_ to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testCompactionWithHFileLink/186d87f306418fb25d295d824c1272fe/family1/13b66e48c9e94749b690ffcbc521a3ef_SeqId_29_ 2023-02-11 03:55:37,566 DEBUG [RestoreSnapshot-pool-2] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/621cd59809a16fd910172a9890d26263/family1/1982d73b09064ef3b63505c41585342c_SeqId_29_ to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testCompactionWithHFileLink/621cd59809a16fd910172a9890d26263/family1/1982d73b09064ef3b63505c41585342c_SeqId_29_ 2023-02-11 03:55:37,566 DEBUG [RestoreSnapshot-pool-2] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink a54c49bae3b34318920fc4197db7ae61 of snapshot=snaptb-1676087720079 to region=621cd59809a16fd910172a9890d26263 table=testCompactionWithHFileLink 2023-02-11 03:55:37,566 DEBUG [RestoreSnapshot-pool-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/ea4735f297efbbfcc4cbc0bd3d6e42e0/family1/a0d705d5ed3447a99303b7d9468cd95b_SeqId_29_ to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testCompactionWithHFileLink/ea4735f297efbbfcc4cbc0bd3d6e42e0/family1/a0d705d5ed3447a99303b7d9468cd95b_SeqId_29_ 2023-02-11 03:55:37,566 DEBUG [RestoreSnapshot-pool-3] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 92743cf8525b4b5d877b358e975a3bec of snapshot=snaptb-1676087720079 to region=ea4735f297efbbfcc4cbc0bd3d6e42e0 table=testCompactionWithHFileLink 2023-02-11 03:55:37,566 DEBUG [RestoreSnapshot-pool-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/186d87f306418fb25d295d824c1272fe/family1/33bb864dff41474f9562086b84b9018e to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testCompactionWithHFileLink/186d87f306418fb25d295d824c1272fe/family1/33bb864dff41474f9562086b84b9018e 2023-02-11 03:55:37,566 DEBUG [RestoreSnapshot-pool-1] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 2e6dcfa9c78543fcb2273f9d67b88e36 of snapshot=snaptb-1676087720079 to region=186d87f306418fb25d295d824c1272fe table=testCompactionWithHFileLink 2023-02-11 03:55:37,569 DEBUG [RestoreSnapshot-pool-2] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 051a49ec8101444f8e462520d66fa021 of snapshot=snaptb-1676087720079 to region=621cd59809a16fd910172a9890d26263 table=testCompactionWithHFileLink 2023-02-11 03:55:37,570 DEBUG [RestoreSnapshot-pool-3] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 0a642f51d6d94a79b4c5663961dee5c0 of snapshot=snaptb-1676087720079 to region=ea4735f297efbbfcc4cbc0bd3d6e42e0 table=testCompactionWithHFileLink 2023-02-11 03:55:37,570 DEBUG [RestoreSnapshot-pool-1] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 4e9479861f4540a6b8a67dee5f4b8c2c of snapshot=snaptb-1676087720079 to region=186d87f306418fb25d295d824c1272fe table=testCompactionWithHFileLink 2023-02-11 03:55:37,575 DEBUG [RestoreSnapshot-pool-3] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c504159ab71244858e1768f8eb16927b of snapshot=snaptb-1676087720079 to region=ea4735f297efbbfcc4cbc0bd3d6e42e0 table=testCompactionWithHFileLink 2023-02-11 03:55:37,575 DEBUG [RestoreSnapshot-pool-2] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 3cb4dc3b66114935967cd14111320066 of snapshot=snaptb-1676087720079 to region=621cd59809a16fd910172a9890d26263 table=testCompactionWithHFileLink 2023-02-11 03:55:37,575 DEBUG [RestoreSnapshot-pool-1] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 55bf25ed3b544fe9969ac6430989cefc of snapshot=snaptb-1676087720079 to region=186d87f306418fb25d295d824c1272fe table=testCompactionWithHFileLink 2023-02-11 03:55:37,582 DEBUG [RestoreSnapshot-pool-3] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 3088ce87ec264e38859de9f824a4ab99 of snapshot=snaptb-1676087720079 to region=ea4735f297efbbfcc4cbc0bd3d6e42e0 table=testCompactionWithHFileLink 2023-02-11 03:55:37,582 DEBUG [RestoreSnapshot-pool-2] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 42ea1ea361834322b07cb7fa95ee893f of snapshot=snaptb-1676087720079 to region=621cd59809a16fd910172a9890d26263 table=testCompactionWithHFileLink 2023-02-11 03:55:37,582 DEBUG [RestoreSnapshot-pool-1] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 1363e2a6dd1f49e0a9d550f758d2bcac of snapshot=snaptb-1676087720079 to region=186d87f306418fb25d295d824c1272fe table=testCompactionWithHFileLink 2023-02-11 03:55:37,595 DEBUG [RestoreSnapshot-pool-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/186d87f306418fb25d295d824c1272fe/family2/6046f3d6ed634b868e201512cc6356ce to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testCompactionWithHFileLink/186d87f306418fb25d295d824c1272fe/family2/6046f3d6ed634b868e201512cc6356ce 2023-02-11 03:55:37,595 DEBUG [RestoreSnapshot-pool-1] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 48173b765b124663a97b2f43b90e8a88 of snapshot=snaptb-1676087720079 to region=186d87f306418fb25d295d824c1272fe table=testCompactionWithHFileLink 2023-02-11 03:55:37,595 DEBUG [RestoreSnapshot-pool-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/ea4735f297efbbfcc4cbc0bd3d6e42e0/family2/bccb725e01334e4b8e122c684ffc0e2e to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testCompactionWithHFileLink/ea4735f297efbbfcc4cbc0bd3d6e42e0/family2/bccb725e01334e4b8e122c684ffc0e2e 2023-02-11 03:55:37,595 DEBUG [RestoreSnapshot-pool-3] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 7141c179434b4a0f816573a88da1caf7 of snapshot=snaptb-1676087720079 to region=ea4735f297efbbfcc4cbc0bd3d6e42e0 table=testCompactionWithHFileLink 2023-02-11 03:55:37,596 DEBUG [RestoreSnapshot-pool-2] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/621cd59809a16fd910172a9890d26263/family2/8e6e96733be0435d8cd67d27596328ca to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testCompactionWithHFileLink/621cd59809a16fd910172a9890d26263/family2/8e6e96733be0435d8cd67d27596328ca 2023-02-11 03:55:37,596 DEBUG [RestoreSnapshot-pool-2] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 774f5f8b4d2e463f983d817bdedf591e of snapshot=snaptb-1676087720079 to region=621cd59809a16fd910172a9890d26263 table=testCompactionWithHFileLink 2023-02-11 03:55:37,606 DEBUG [RestoreSnapshot-pool-1] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 039e3e0c12c742f29c5ea86a66e98c44 of snapshot=snaptb-1676087720079 to region=186d87f306418fb25d295d824c1272fe table=testCompactionWithHFileLink 2023-02-11 03:55:37,606 DEBUG [RestoreSnapshot-pool-2] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink e000a979eed34f31ba1476e0d7c949da of snapshot=snaptb-1676087720079 to region=621cd59809a16fd910172a9890d26263 table=testCompactionWithHFileLink 2023-02-11 03:55:37,606 DEBUG [RestoreSnapshot-pool-3] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 045314dc19574fe9979fbedc8e14acf8 of snapshot=snaptb-1676087720079 to region=ea4735f297efbbfcc4cbc0bd3d6e42e0 table=testCompactionWithHFileLink 2023-02-11 03:55:37,609 DEBUG [RestoreSnapshot-pool-1] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 14bfe13c870f426d986ef125637ce8f4 of snapshot=snaptb-1676087720079 to region=186d87f306418fb25d295d824c1272fe table=testCompactionWithHFileLink 2023-02-11 03:55:37,609 DEBUG [RestoreSnapshot-pool-2] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 87ccd53b473f47b6887f864a901e3b7c of snapshot=snaptb-1676087720079 to region=621cd59809a16fd910172a9890d26263 table=testCompactionWithHFileLink 2023-02-11 03:55:37,609 DEBUG [RestoreSnapshot-pool-3] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink a77823d8bc254eb5b1130f536fa90187 of snapshot=snaptb-1676087720079 to region=ea4735f297efbbfcc4cbc0bd3d6e42e0 table=testCompactionWithHFileLink 2023-02-11 03:55:37,612 DEBUG [RestoreSnapshot-pool-1] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 203bb908b48a4908b252cec8e0b7c5e7 of snapshot=snaptb-1676087720079 to region=186d87f306418fb25d295d824c1272fe table=testCompactionWithHFileLink 2023-02-11 03:55:37,612 DEBUG [RestoreSnapshot-pool-2] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 32530f87de8648c0ab8a83636ca1711f of snapshot=snaptb-1676087720079 to region=621cd59809a16fd910172a9890d26263 table=testCompactionWithHFileLink 2023-02-11 03:55:37,612 DEBUG [RestoreSnapshot-pool-3] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 5b41b5f362864186bb9f20144e3bc4c5 of snapshot=snaptb-1676087720079 to region=ea4735f297efbbfcc4cbc0bd3d6e42e0 table=testCompactionWithHFileLink 2023-02-11 03:55:37,616 INFO [PEWorker-2] snapshot.RestoreSnapshotHelper(266): finishing restore table regions using snapshot=name: "snaptb-1676087720079" table: "testCompactionWithHFileLink" creation_time: 1676087720089 type: FLUSH version: 2 ttl: 0 2023-02-11 03:55:37,617 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testCompactionWithHFileLink,,1676087711806.186d87f306418fb25d295d824c1272fe.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676087737617"}]},"ts":"9223372036854775807"} 2023-02-11 03:55:37,618 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testCompactionWithHFileLink,2,1676087711806.621cd59809a16fd910172a9890d26263.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676087737617"}]},"ts":"9223372036854775807"} 2023-02-11 03:55:37,618 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testCompactionWithHFileLink,1,1676087711806.ea4735f297efbbfcc4cbc0bd3d6e42e0.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676087737617"}]},"ts":"9223372036854775807"} 2023-02-11 03:55:37,619 INFO [PEWorker-2] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-02-11 03:55:37,620 DEBUG [PEWorker-2] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 186d87f306418fb25d295d824c1272fe, NAME => 'testCompactionWithHFileLink,,1676087711806.186d87f306418fb25d295d824c1272fe.', STARTKEY => '', ENDKEY => '1'}, {ENCODED => 621cd59809a16fd910172a9890d26263, NAME => 'testCompactionWithHFileLink,2,1676087711806.621cd59809a16fd910172a9890d26263.', STARTKEY => '2', ENDKEY => ''}, {ENCODED => ea4735f297efbbfcc4cbc0bd3d6e42e0, NAME => 'testCompactionWithHFileLink,1,1676087711806.ea4735f297efbbfcc4cbc0bd3d6e42e0.', STARTKEY => '1', ENDKEY => '2'}] 2023-02-11 03:55:37,620 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,,1676087711806.186d87f306418fb25d295d824c1272fe.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1676087737618"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676087737618"}]},"ts":"1676087737618"} 2023-02-11 03:55:37,620 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,2,1676087711806.621cd59809a16fd910172a9890d26263.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1676087737618"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676087737618"}]},"ts":"1676087737618"} 2023-02-11 03:55:37,620 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,1,1676087711806.ea4735f297efbbfcc4cbc0bd3d6e42e0.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1676087737618"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676087737618"}]},"ts":"1676087737618"} 2023-02-11 03:55:37,621 INFO [PEWorker-2] hbase.MetaTableAccessor(1496): Added 3 regions to meta. 2023-02-11 03:55:37,621 INFO [PEWorker-2] hbase.MetaTableAccessor(1810): Overwritten 3 regions to Meta 2023-02-11 03:55:37,621 DEBUG [PEWorker-2] hbase.MetaTableAccessor(1811): Overwritten regions: [{ENCODED => 186d87f306418fb25d295d824c1272fe, NAME => 'testCompactionWithHFileLink,,1676087711806.186d87f306418fb25d295d824c1272fe.', STARTKEY => '', ENDKEY => '1'}, {ENCODED => 621cd59809a16fd910172a9890d26263, NAME => 'testCompactionWithHFileLink,2,1676087711806.621cd59809a16fd910172a9890d26263.', STARTKEY => '2', ENDKEY => ''}, {ENCODED => ea4735f297efbbfcc4cbc0bd3d6e42e0, NAME => 'testCompactionWithHFileLink,1,1676087711806.ea4735f297efbbfcc4cbc0bd3d6e42e0.', STARTKEY => '1', ENDKEY => '2'}] 2023-02-11 03:55:37,622 INFO [PEWorker-2] procedure.RestoreSnapshotProcedure(447): Restore snapshot={ ss=snaptb-1676087720079 table=testCompactionWithHFileLink type=FLUSH ttl=0 } on table=testCompactionWithHFileLink completed! 2023-02-11 03:55:37,623 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=96, state=SUCCESS; RestoreSnapshotProcedure (table=testCompactionWithHFileLink snapshot=name: "snaptb-1676087720079" table: "testCompactionWithHFileLink" creation_time: 1676087720089 type: FLUSH version: 2 ttl: 0 ) in 534 msec 2023-02-11 03:55:37,703 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39921] master.MasterRpcServices(1227): Checking to see if procedure is done pid=96 2023-02-11 03:55:37,703 INFO [Listener at localhost/46205] client.HBaseAdmin$TableFuture(3541): Operation: MODIFY, Table Name: default:testCompactionWithHFileLink, procId: 96 completed 2023-02-11 03:55:37,703 INFO [Listener at localhost/46205] client.HBaseAdmin(2618): Deleting restore-failsafe snapshot: hbase-failsafe-snaptb-1676087720079-1676087734937 2023-02-11 03:55:37,707 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39921] master.MasterRpcServices(748): Client=jenkins//172.31.14.131 delete name: "hbase-failsafe-snaptb-1676087720079-1676087734937" 2023-02-11 03:55:37,709 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39921] snapshot.SnapshotManager(333): Deleting snapshot: hbase-failsafe-snaptb-1676087720079-1676087734937 2023-02-11 03:55:37,711 INFO [Listener at localhost/46205] client.HBaseAdmin$14(830): Started enable of testCompactionWithHFileLink 2023-02-11 03:55:37,715 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39921] master.HMaster$10(2372): Client=jenkins//172.31.14.131 enable testCompactionWithHFileLink 2023-02-11 03:55:37,721 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39921] procedure2.ProcedureExecutor(1029): Stored pid=97, state=RUNNABLE:ENABLE_TABLE_PREPARE; EnableTableProcedure table=testCompactionWithHFileLink 2023-02-11 03:55:37,724 INFO [PEWorker-4] procedure.EnableTableProcedure(296): Attempting to enable the table testCompactionWithHFileLink 2023-02-11 03:55:37,724 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39921] master.MasterRpcServices(1227): Checking to see if procedure is done pid=97 2023-02-11 03:55:37,724 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testCompactionWithHFileLink","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676087737724"}]},"ts":"1676087737724"} 2023-02-11 03:55:37,725 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=testCompactionWithHFileLink, state=ENABLING in hbase:meta 2023-02-11 03:55:37,729 DEBUG [PEWorker-4] procedure.EnableTableProcedure(107): No change in number of region replicas (configuredReplicaCount=1); assigning. 2023-02-11 03:55:37,729 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=98, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=186d87f306418fb25d295d824c1272fe, ASSIGN}, {pid=99, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=ea4735f297efbbfcc4cbc0bd3d6e42e0, ASSIGN}, {pid=100, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=621cd59809a16fd910172a9890d26263, ASSIGN}] 2023-02-11 03:55:37,730 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=100, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=621cd59809a16fd910172a9890d26263, ASSIGN 2023-02-11 03:55:37,730 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=99, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=ea4735f297efbbfcc4cbc0bd3d6e42e0, ASSIGN 2023-02-11 03:55:37,730 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=98, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=186d87f306418fb25d295d824c1272fe, ASSIGN 2023-02-11 03:55:37,731 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=100, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=621cd59809a16fd910172a9890d26263, ASSIGN; state=CLOSED, location=null; forceNewPlan=false, retain=false 2023-02-11 03:55:37,731 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=99, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=ea4735f297efbbfcc4cbc0bd3d6e42e0, ASSIGN; state=CLOSED, location=null; forceNewPlan=false, retain=false 2023-02-11 03:55:37,731 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=98, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=186d87f306418fb25d295d824c1272fe, ASSIGN; state=CLOSED, location=null; forceNewPlan=false, retain=false 2023-02-11 03:55:37,825 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39921] master.MasterRpcServices(1227): Checking to see if procedure is done pid=97 2023-02-11 03:55:37,882 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=100 updating hbase:meta row=621cd59809a16fd910172a9890d26263, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:55:37,882 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=98 updating hbase:meta row=186d87f306418fb25d295d824c1272fe, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:55:37,882 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=99 updating hbase:meta row=ea4735f297efbbfcc4cbc0bd3d6e42e0, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:55:37,882 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,2,1676087711806.621cd59809a16fd910172a9890d26263.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1676087737882"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676087737882"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676087737882"}]},"ts":"1676087737882"} 2023-02-11 03:55:37,882 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,,1676087711806.186d87f306418fb25d295d824c1272fe.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1676087737882"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676087737882"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676087737882"}]},"ts":"1676087737882"} 2023-02-11 03:55:37,882 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,1,1676087711806.ea4735f297efbbfcc4cbc0bd3d6e42e0.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1676087737882"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676087737882"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676087737882"}]},"ts":"1676087737882"} 2023-02-11 03:55:37,884 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=101, ppid=100, state=RUNNABLE; OpenRegionProcedure 621cd59809a16fd910172a9890d26263, server=jenkins-hbase4.apache.org,37011,1676087618428}] 2023-02-11 03:55:37,884 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=102, ppid=98, state=RUNNABLE; OpenRegionProcedure 186d87f306418fb25d295d824c1272fe, server=jenkins-hbase4.apache.org,37011,1676087618428}] 2023-02-11 03:55:37,885 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=103, ppid=99, state=RUNNABLE; OpenRegionProcedure ea4735f297efbbfcc4cbc0bd3d6e42e0, server=jenkins-hbase4.apache.org,37011,1676087618428}] 2023-02-11 03:55:38,026 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39921] master.MasterRpcServices(1227): Checking to see if procedure is done pid=97 2023-02-11 03:55:38,038 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testCompactionWithHFileLink,,1676087711806.186d87f306418fb25d295d824c1272fe. 2023-02-11 03:55:38,038 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 186d87f306418fb25d295d824c1272fe, NAME => 'testCompactionWithHFileLink,,1676087711806.186d87f306418fb25d295d824c1272fe.', STARTKEY => '', ENDKEY => '1'} 2023-02-11 03:55:38,038 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testCompactionWithHFileLink 186d87f306418fb25d295d824c1272fe 2023-02-11 03:55:38,038 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testCompactionWithHFileLink,,1676087711806.186d87f306418fb25d295d824c1272fe.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-11 03:55:38,038 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 186d87f306418fb25d295d824c1272fe 2023-02-11 03:55:38,038 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 186d87f306418fb25d295d824c1272fe 2023-02-11 03:55:38,039 INFO [StoreOpener-186d87f306418fb25d295d824c1272fe-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 186d87f306418fb25d295d824c1272fe 2023-02-11 03:55:38,040 DEBUG [StoreOpener-186d87f306418fb25d295d824c1272fe-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/186d87f306418fb25d295d824c1272fe/family1 2023-02-11 03:55:38,040 DEBUG [StoreOpener-186d87f306418fb25d295d824c1272fe-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/186d87f306418fb25d295d824c1272fe/family1 2023-02-11 03:55:38,041 INFO [StoreOpener-186d87f306418fb25d295d824c1272fe-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 186d87f306418fb25d295d824c1272fe columnFamilyName family1 2023-02-11 03:55:38,055 DEBUG [StoreOpener-186d87f306418fb25d295d824c1272fe-1] regionserver.HStore(539): loaded hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/186d87f306418fb25d295d824c1272fe/family1/testCompactionWithHFileLink=186d87f306418fb25d295d824c1272fe-1363e2a6dd1f49e0a9d550f758d2bcac 2023-02-11 03:55:38,060 DEBUG [StoreOpener-186d87f306418fb25d295d824c1272fe-1] regionserver.HStore(539): loaded hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/186d87f306418fb25d295d824c1272fe/family1/testCompactionWithHFileLink=186d87f306418fb25d295d824c1272fe-2e6dcfa9c78543fcb2273f9d67b88e36 2023-02-11 03:55:38,065 DEBUG [StoreOpener-186d87f306418fb25d295d824c1272fe-1] regionserver.HStore(539): loaded hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/186d87f306418fb25d295d824c1272fe/family1/testCompactionWithHFileLink=186d87f306418fb25d295d824c1272fe-4e9479861f4540a6b8a67dee5f4b8c2c 2023-02-11 03:55:38,069 DEBUG [StoreOpener-186d87f306418fb25d295d824c1272fe-1] regionserver.HStore(539): loaded hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/186d87f306418fb25d295d824c1272fe/family1/testCompactionWithHFileLink=186d87f306418fb25d295d824c1272fe-55bf25ed3b544fe9969ac6430989cefc 2023-02-11 03:55:38,069 INFO [StoreOpener-186d87f306418fb25d295d824c1272fe-1] regionserver.HStore(310): Store=186d87f306418fb25d295d824c1272fe/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 03:55:38,070 INFO [StoreOpener-186d87f306418fb25d295d824c1272fe-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 186d87f306418fb25d295d824c1272fe 2023-02-11 03:55:38,070 DEBUG [StoreOpener-186d87f306418fb25d295d824c1272fe-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/186d87f306418fb25d295d824c1272fe/family2 2023-02-11 03:55:38,070 DEBUG [StoreOpener-186d87f306418fb25d295d824c1272fe-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/186d87f306418fb25d295d824c1272fe/family2 2023-02-11 03:55:38,071 INFO [StoreOpener-186d87f306418fb25d295d824c1272fe-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 186d87f306418fb25d295d824c1272fe columnFamilyName family2 2023-02-11 03:55:38,078 DEBUG [StoreOpener-186d87f306418fb25d295d824c1272fe-1] regionserver.HStore(539): loaded hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/186d87f306418fb25d295d824c1272fe/family2/testCompactionWithHFileLink=186d87f306418fb25d295d824c1272fe-039e3e0c12c742f29c5ea86a66e98c44 2023-02-11 03:55:38,083 DEBUG [StoreOpener-186d87f306418fb25d295d824c1272fe-1] regionserver.HStore(539): loaded hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/186d87f306418fb25d295d824c1272fe/family2/testCompactionWithHFileLink=186d87f306418fb25d295d824c1272fe-14bfe13c870f426d986ef125637ce8f4 2023-02-11 03:55:38,088 DEBUG [StoreOpener-186d87f306418fb25d295d824c1272fe-1] regionserver.HStore(539): loaded hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/186d87f306418fb25d295d824c1272fe/family2/testCompactionWithHFileLink=186d87f306418fb25d295d824c1272fe-203bb908b48a4908b252cec8e0b7c5e7 2023-02-11 03:55:38,093 DEBUG [StoreOpener-186d87f306418fb25d295d824c1272fe-1] regionserver.HStore(539): loaded hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/186d87f306418fb25d295d824c1272fe/family2/testCompactionWithHFileLink=186d87f306418fb25d295d824c1272fe-48173b765b124663a97b2f43b90e8a88 2023-02-11 03:55:38,093 INFO [StoreOpener-186d87f306418fb25d295d824c1272fe-1] regionserver.HStore(310): Store=186d87f306418fb25d295d824c1272fe/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 03:55:38,094 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/186d87f306418fb25d295d824c1272fe 2023-02-11 03:55:38,095 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/186d87f306418fb25d295d824c1272fe 2023-02-11 03:55:38,096 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testCompactionWithHFileLink descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-11 03:55:38,097 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 186d87f306418fb25d295d824c1272fe 2023-02-11 03:55:38,098 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 186d87f306418fb25d295d824c1272fe; next sequenceid=32; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=73893786, jitterRate=0.10110321640968323}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-11 03:55:38,098 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 186d87f306418fb25d295d824c1272fe: Running coprocessor pre-open hook at 1676087738038Writing region info on filesystem at 1676087738038Initializing all the Stores at 1676087738039 (+1 ms)Instantiating store for column family {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1676087738039Instantiating store for column family {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1676087738039Cleaning up temporary data from old regions at 1676087738095 (+56 ms)Cleaning up detritus from prior splits at 1676087738096 (+1 ms)Running coprocessor post-open hooks at 1676087738098 (+2 ms)Region opened successfully at 1676087738098 2023-02-11 03:55:38,099 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testCompactionWithHFileLink,,1676087711806.186d87f306418fb25d295d824c1272fe., pid=102, masterSystemTime=1676087738035 2023-02-11 03:55:38,099 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-11 03:55:38,099 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-11 03:55:38,099 DEBUG [RS:0;jenkins-hbase4:37011-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 4 store files, 0 compacting, 4 eligible, 16 blocking 2023-02-11 03:55:38,099 DEBUG [RS:0;jenkins-hbase4:37011-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 4 store files, 0 compacting, 4 eligible, 16 blocking 2023-02-11 03:55:38,101 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testCompactionWithHFileLink,,1676087711806.186d87f306418fb25d295d824c1272fe. 2023-02-11 03:55:38,101 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testCompactionWithHFileLink,,1676087711806.186d87f306418fb25d295d824c1272fe. 2023-02-11 03:55:38,101 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testCompactionWithHFileLink,2,1676087711806.621cd59809a16fd910172a9890d26263. 2023-02-11 03:55:38,101 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=98 updating hbase:meta row=186d87f306418fb25d295d824c1272fe, regionState=OPEN, openSeqNum=32, regionLocation=jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:55:38,101 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 621cd59809a16fd910172a9890d26263, NAME => 'testCompactionWithHFileLink,2,1676087711806.621cd59809a16fd910172a9890d26263.', STARTKEY => '2', ENDKEY => ''} 2023-02-11 03:55:38,101 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testCompactionWithHFileLink,,1676087711806.186d87f306418fb25d295d824c1272fe.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1676087738101"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676087738101"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676087738101"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676087738101"}]},"ts":"1676087738101"} 2023-02-11 03:55:38,101 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testCompactionWithHFileLink 621cd59809a16fd910172a9890d26263 2023-02-11 03:55:38,102 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testCompactionWithHFileLink,2,1676087711806.621cd59809a16fd910172a9890d26263.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-11 03:55:38,102 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 621cd59809a16fd910172a9890d26263 2023-02-11 03:55:38,102 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 621cd59809a16fd910172a9890d26263 2023-02-11 03:55:38,103 DEBUG [RS:0;jenkins-hbase4:37011-longCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 4 files of size 20920 starting at candidate #0 after considering 3 permutations with 3 in ratio 2023-02-11 03:55:38,103 DEBUG [RS:0;jenkins-hbase4:37011-longCompactions-0] regionserver.HStore(1912): 186d87f306418fb25d295d824c1272fe/family2 is initiating minor compaction (all files) 2023-02-11 03:55:38,103 DEBUG [RS:0;jenkins-hbase4:37011-shortCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 4 files of size 22152 starting at candidate #0 after considering 3 permutations with 3 in ratio 2023-02-11 03:55:38,103 INFO [RS:0;jenkins-hbase4:37011-longCompactions-0] regionserver.HRegion(2258): Starting compaction of 186d87f306418fb25d295d824c1272fe/family2 in testCompactionWithHFileLink,,1676087711806.186d87f306418fb25d295d824c1272fe. 2023-02-11 03:55:38,103 DEBUG [RS:0;jenkins-hbase4:37011-shortCompactions-0] regionserver.HStore(1912): 186d87f306418fb25d295d824c1272fe/family1 is initiating minor compaction (all files) 2023-02-11 03:55:38,103 INFO [RS:0;jenkins-hbase4:37011-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/186d87f306418fb25d295d824c1272fe/family2/testCompactionWithHFileLink=186d87f306418fb25d295d824c1272fe-48173b765b124663a97b2f43b90e8a88, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/186d87f306418fb25d295d824c1272fe/family2/testCompactionWithHFileLink=186d87f306418fb25d295d824c1272fe-039e3e0c12c742f29c5ea86a66e98c44, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/186d87f306418fb25d295d824c1272fe/family2/testCompactionWithHFileLink=186d87f306418fb25d295d824c1272fe-14bfe13c870f426d986ef125637ce8f4, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/186d87f306418fb25d295d824c1272fe/family2/testCompactionWithHFileLink=186d87f306418fb25d295d824c1272fe-203bb908b48a4908b252cec8e0b7c5e7] into tmpdir=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/186d87f306418fb25d295d824c1272fe/.tmp, totalSize=20.4 K 2023-02-11 03:55:38,103 INFO [RS:0;jenkins-hbase4:37011-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 186d87f306418fb25d295d824c1272fe/family1 in testCompactionWithHFileLink,,1676087711806.186d87f306418fb25d295d824c1272fe. 2023-02-11 03:55:38,103 INFO [RS:0;jenkins-hbase4:37011-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/186d87f306418fb25d295d824c1272fe/family1/testCompactionWithHFileLink=186d87f306418fb25d295d824c1272fe-2e6dcfa9c78543fcb2273f9d67b88e36, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/186d87f306418fb25d295d824c1272fe/family1/testCompactionWithHFileLink=186d87f306418fb25d295d824c1272fe-4e9479861f4540a6b8a67dee5f4b8c2c, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/186d87f306418fb25d295d824c1272fe/family1/testCompactionWithHFileLink=186d87f306418fb25d295d824c1272fe-55bf25ed3b544fe9969ac6430989cefc, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/186d87f306418fb25d295d824c1272fe/family1/testCompactionWithHFileLink=186d87f306418fb25d295d824c1272fe-1363e2a6dd1f49e0a9d550f758d2bcac] into tmpdir=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/186d87f306418fb25d295d824c1272fe/.tmp, totalSize=21.6 K 2023-02-11 03:55:38,104 INFO [StoreOpener-621cd59809a16fd910172a9890d26263-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 621cd59809a16fd910172a9890d26263 2023-02-11 03:55:38,104 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=102, resume processing ppid=98 2023-02-11 03:55:38,104 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=102, ppid=98, state=SUCCESS; OpenRegionProcedure 186d87f306418fb25d295d824c1272fe, server=jenkins-hbase4.apache.org,37011,1676087618428 in 219 msec 2023-02-11 03:55:38,104 DEBUG [RS:0;jenkins-hbase4:37011-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=186d87f306418fb25d295d824c1272fe-48173b765b124663a97b2f43b90e8a88, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1676087712920 2023-02-11 03:55:38,104 DEBUG [RS:0;jenkins-hbase4:37011-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=186d87f306418fb25d295d824c1272fe-2e6dcfa9c78543fcb2273f9d67b88e36, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1676087712920 2023-02-11 03:55:38,105 DEBUG [RS:0;jenkins-hbase4:37011-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=186d87f306418fb25d295d824c1272fe-039e3e0c12c742f29c5ea86a66e98c44, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1676087714701 2023-02-11 03:55:38,105 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=98, ppid=97, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=186d87f306418fb25d295d824c1272fe, ASSIGN in 375 msec 2023-02-11 03:55:38,105 DEBUG [RS:0;jenkins-hbase4:37011-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=186d87f306418fb25d295d824c1272fe-4e9479861f4540a6b8a67dee5f4b8c2c, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1676087714701 2023-02-11 03:55:38,105 DEBUG [StoreOpener-621cd59809a16fd910172a9890d26263-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/621cd59809a16fd910172a9890d26263/family1 2023-02-11 03:55:38,105 DEBUG [StoreOpener-621cd59809a16fd910172a9890d26263-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/621cd59809a16fd910172a9890d26263/family1 2023-02-11 03:55:38,106 INFO [StoreOpener-621cd59809a16fd910172a9890d26263-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 621cd59809a16fd910172a9890d26263 columnFamilyName family1 2023-02-11 03:55:38,106 DEBUG [RS:0;jenkins-hbase4:37011-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=186d87f306418fb25d295d824c1272fe-14bfe13c870f426d986ef125637ce8f4, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1676087716494 2023-02-11 03:55:38,106 DEBUG [RS:0;jenkins-hbase4:37011-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=186d87f306418fb25d295d824c1272fe-55bf25ed3b544fe9969ac6430989cefc, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1676087716494 2023-02-11 03:55:38,107 DEBUG [RS:0;jenkins-hbase4:37011-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=186d87f306418fb25d295d824c1272fe-203bb908b48a4908b252cec8e0b7c5e7, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1676087718286 2023-02-11 03:55:38,107 DEBUG [RS:0;jenkins-hbase4:37011-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=186d87f306418fb25d295d824c1272fe-1363e2a6dd1f49e0a9d550f758d2bcac, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1676087718286 2023-02-11 03:55:38,119 DEBUG [StoreOpener-621cd59809a16fd910172a9890d26263-1] regionserver.HStore(539): loaded hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/621cd59809a16fd910172a9890d26263/family1/testCompactionWithHFileLink=621cd59809a16fd910172a9890d26263-051a49ec8101444f8e462520d66fa021 2023-02-11 03:55:38,134 DEBUG [StoreOpener-621cd59809a16fd910172a9890d26263-1] regionserver.HStore(539): loaded hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/621cd59809a16fd910172a9890d26263/family1/testCompactionWithHFileLink=621cd59809a16fd910172a9890d26263-3cb4dc3b66114935967cd14111320066 2023-02-11 03:55:38,147 DEBUG [StoreOpener-621cd59809a16fd910172a9890d26263-1] regionserver.HStore(539): loaded hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/621cd59809a16fd910172a9890d26263/family1/testCompactionWithHFileLink=621cd59809a16fd910172a9890d26263-42ea1ea361834322b07cb7fa95ee893f 2023-02-11 03:55:38,151 INFO [RS:0;jenkins-hbase4:37011-longCompactions-0] throttle.PressureAwareThroughputController(145): 186d87f306418fb25d295d824c1272fe#family2#compaction#113 average throughput is 0.39 MB/second, slept 0 time(s) and total slept time is 0 ms. 1 active operations remaining, total limit is 61.54 MB/second 2023-02-11 03:55:38,153 INFO [RS:0;jenkins-hbase4:37011-shortCompactions-0] throttle.PressureAwareThroughputController(145): 186d87f306418fb25d295d824c1272fe#family1#compaction#114 average throughput is 0.78 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 61.54 MB/second 2023-02-11 03:55:38,156 DEBUG [StoreOpener-621cd59809a16fd910172a9890d26263-1] regionserver.HStore(539): loaded hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/621cd59809a16fd910172a9890d26263/family1/testCompactionWithHFileLink=621cd59809a16fd910172a9890d26263-a54c49bae3b34318920fc4197db7ae61 2023-02-11 03:55:38,156 INFO [StoreOpener-621cd59809a16fd910172a9890d26263-1] regionserver.HStore(310): Store=621cd59809a16fd910172a9890d26263/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 03:55:38,156 INFO [StoreOpener-621cd59809a16fd910172a9890d26263-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 621cd59809a16fd910172a9890d26263 2023-02-11 03:55:38,157 DEBUG [StoreOpener-621cd59809a16fd910172a9890d26263-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/621cd59809a16fd910172a9890d26263/family2 2023-02-11 03:55:38,157 DEBUG [StoreOpener-621cd59809a16fd910172a9890d26263-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/621cd59809a16fd910172a9890d26263/family2 2023-02-11 03:55:38,157 INFO [StoreOpener-621cd59809a16fd910172a9890d26263-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 621cd59809a16fd910172a9890d26263 columnFamilyName family2 2023-02-11 03:55:38,169 DEBUG [StoreOpener-621cd59809a16fd910172a9890d26263-1] regionserver.HStore(539): loaded hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/621cd59809a16fd910172a9890d26263/family2/testCompactionWithHFileLink=621cd59809a16fd910172a9890d26263-32530f87de8648c0ab8a83636ca1711f 2023-02-11 03:55:38,173 DEBUG [StoreOpener-621cd59809a16fd910172a9890d26263-1] regionserver.HStore(539): loaded hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/621cd59809a16fd910172a9890d26263/family2/testCompactionWithHFileLink=621cd59809a16fd910172a9890d26263-774f5f8b4d2e463f983d817bdedf591e 2023-02-11 03:55:38,177 DEBUG [StoreOpener-621cd59809a16fd910172a9890d26263-1] regionserver.HStore(539): loaded hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/621cd59809a16fd910172a9890d26263/family2/testCompactionWithHFileLink=621cd59809a16fd910172a9890d26263-87ccd53b473f47b6887f864a901e3b7c 2023-02-11 03:55:38,181 DEBUG [StoreOpener-621cd59809a16fd910172a9890d26263-1] regionserver.HStore(539): loaded hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/621cd59809a16fd910172a9890d26263/family2/testCompactionWithHFileLink=621cd59809a16fd910172a9890d26263-e000a979eed34f31ba1476e0d7c949da 2023-02-11 03:55:38,181 INFO [StoreOpener-621cd59809a16fd910172a9890d26263-1] regionserver.HStore(310): Store=621cd59809a16fd910172a9890d26263/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 03:55:38,182 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/621cd59809a16fd910172a9890d26263 2023-02-11 03:55:38,183 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/621cd59809a16fd910172a9890d26263 2023-02-11 03:55:38,184 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testCompactionWithHFileLink descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-11 03:55:38,185 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 621cd59809a16fd910172a9890d26263 2023-02-11 03:55:38,186 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 621cd59809a16fd910172a9890d26263; next sequenceid=32; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=64780729, jitterRate=-0.03469191491603851}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-11 03:55:38,186 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 621cd59809a16fd910172a9890d26263: Running coprocessor pre-open hook at 1676087738102Writing region info on filesystem at 1676087738102Initializing all the Stores at 1676087738103 (+1 ms)Instantiating store for column family {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1676087738103Instantiating store for column family {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1676087738104 (+1 ms)Cleaning up temporary data from old regions at 1676087738183 (+79 ms)Cleaning up detritus from prior splits at 1676087738184 (+1 ms)Running coprocessor post-open hooks at 1676087738186 (+2 ms)Region opened successfully at 1676087738186 2023-02-11 03:55:38,187 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testCompactionWithHFileLink,2,1676087711806.621cd59809a16fd910172a9890d26263., pid=101, masterSystemTime=1676087738035 2023-02-11 03:55:38,187 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-11 03:55:38,187 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=2), splitQueue=0 2023-02-11 03:55:38,188 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testCompactionWithHFileLink,2,1676087711806.621cd59809a16fd910172a9890d26263. 2023-02-11 03:55:38,188 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testCompactionWithHFileLink,2,1676087711806.621cd59809a16fd910172a9890d26263. 2023-02-11 03:55:38,188 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testCompactionWithHFileLink,1,1676087711806.ea4735f297efbbfcc4cbc0bd3d6e42e0. 2023-02-11 03:55:38,188 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => ea4735f297efbbfcc4cbc0bd3d6e42e0, NAME => 'testCompactionWithHFileLink,1,1676087711806.ea4735f297efbbfcc4cbc0bd3d6e42e0.', STARTKEY => '1', ENDKEY => '2'} 2023-02-11 03:55:38,188 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testCompactionWithHFileLink ea4735f297efbbfcc4cbc0bd3d6e42e0 2023-02-11 03:55:38,189 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=100 updating hbase:meta row=621cd59809a16fd910172a9890d26263, regionState=OPEN, openSeqNum=32, regionLocation=jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:55:38,189 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testCompactionWithHFileLink,1,1676087711806.ea4735f297efbbfcc4cbc0bd3d6e42e0.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-11 03:55:38,189 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for ea4735f297efbbfcc4cbc0bd3d6e42e0 2023-02-11 03:55:38,189 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for ea4735f297efbbfcc4cbc0bd3d6e42e0 2023-02-11 03:55:38,189 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testCompactionWithHFileLink,2,1676087711806.621cd59809a16fd910172a9890d26263.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1676087738188"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676087738188"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676087738188"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676087738188"}]},"ts":"1676087738188"} 2023-02-11 03:55:38,190 INFO [StoreOpener-ea4735f297efbbfcc4cbc0bd3d6e42e0-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region ea4735f297efbbfcc4cbc0bd3d6e42e0 2023-02-11 03:55:38,191 DEBUG [StoreOpener-ea4735f297efbbfcc4cbc0bd3d6e42e0-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/ea4735f297efbbfcc4cbc0bd3d6e42e0/family1 2023-02-11 03:55:38,191 DEBUG [StoreOpener-ea4735f297efbbfcc4cbc0bd3d6e42e0-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/ea4735f297efbbfcc4cbc0bd3d6e42e0/family1 2023-02-11 03:55:38,191 INFO [StoreOpener-ea4735f297efbbfcc4cbc0bd3d6e42e0-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region ea4735f297efbbfcc4cbc0bd3d6e42e0 columnFamilyName family1 2023-02-11 03:55:38,191 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=101, resume processing ppid=100 2023-02-11 03:55:38,191 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=101, ppid=100, state=SUCCESS; OpenRegionProcedure 621cd59809a16fd910172a9890d26263, server=jenkins-hbase4.apache.org,37011,1676087618428 in 306 msec 2023-02-11 03:55:38,192 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=100, ppid=97, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=621cd59809a16fd910172a9890d26263, ASSIGN in 462 msec 2023-02-11 03:55:38,200 DEBUG [StoreOpener-ea4735f297efbbfcc4cbc0bd3d6e42e0-1] regionserver.HStore(539): loaded hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/ea4735f297efbbfcc4cbc0bd3d6e42e0/family1/testCompactionWithHFileLink=ea4735f297efbbfcc4cbc0bd3d6e42e0-0a642f51d6d94a79b4c5663961dee5c0 2023-02-11 03:55:38,207 DEBUG [StoreOpener-ea4735f297efbbfcc4cbc0bd3d6e42e0-1] regionserver.HStore(539): loaded hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/ea4735f297efbbfcc4cbc0bd3d6e42e0/family1/testCompactionWithHFileLink=ea4735f297efbbfcc4cbc0bd3d6e42e0-3088ce87ec264e38859de9f824a4ab99 2023-02-11 03:55:38,213 DEBUG [StoreOpener-ea4735f297efbbfcc4cbc0bd3d6e42e0-1] regionserver.HStore(539): loaded hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/ea4735f297efbbfcc4cbc0bd3d6e42e0/family1/testCompactionWithHFileLink=ea4735f297efbbfcc4cbc0bd3d6e42e0-92743cf8525b4b5d877b358e975a3bec 2023-02-11 03:55:38,219 DEBUG [StoreOpener-ea4735f297efbbfcc4cbc0bd3d6e42e0-1] regionserver.HStore(539): loaded hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/ea4735f297efbbfcc4cbc0bd3d6e42e0/family1/testCompactionWithHFileLink=ea4735f297efbbfcc4cbc0bd3d6e42e0-c504159ab71244858e1768f8eb16927b 2023-02-11 03:55:38,219 INFO [StoreOpener-ea4735f297efbbfcc4cbc0bd3d6e42e0-1] regionserver.HStore(310): Store=ea4735f297efbbfcc4cbc0bd3d6e42e0/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 03:55:38,219 INFO [StoreOpener-ea4735f297efbbfcc4cbc0bd3d6e42e0-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region ea4735f297efbbfcc4cbc0bd3d6e42e0 2023-02-11 03:55:38,220 DEBUG [StoreOpener-ea4735f297efbbfcc4cbc0bd3d6e42e0-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/ea4735f297efbbfcc4cbc0bd3d6e42e0/family2 2023-02-11 03:55:38,220 DEBUG [StoreOpener-ea4735f297efbbfcc4cbc0bd3d6e42e0-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/ea4735f297efbbfcc4cbc0bd3d6e42e0/family2 2023-02-11 03:55:38,220 INFO [StoreOpener-ea4735f297efbbfcc4cbc0bd3d6e42e0-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region ea4735f297efbbfcc4cbc0bd3d6e42e0 columnFamilyName family2 2023-02-11 03:55:38,231 DEBUG [StoreOpener-ea4735f297efbbfcc4cbc0bd3d6e42e0-1] regionserver.HStore(539): loaded hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/ea4735f297efbbfcc4cbc0bd3d6e42e0/family2/testCompactionWithHFileLink=ea4735f297efbbfcc4cbc0bd3d6e42e0-045314dc19574fe9979fbedc8e14acf8 2023-02-11 03:55:38,237 DEBUG [StoreOpener-ea4735f297efbbfcc4cbc0bd3d6e42e0-1] regionserver.HStore(539): loaded hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/ea4735f297efbbfcc4cbc0bd3d6e42e0/family2/testCompactionWithHFileLink=ea4735f297efbbfcc4cbc0bd3d6e42e0-5b41b5f362864186bb9f20144e3bc4c5 2023-02-11 03:55:38,243 DEBUG [StoreOpener-ea4735f297efbbfcc4cbc0bd3d6e42e0-1] regionserver.HStore(539): loaded hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/ea4735f297efbbfcc4cbc0bd3d6e42e0/family2/testCompactionWithHFileLink=ea4735f297efbbfcc4cbc0bd3d6e42e0-7141c179434b4a0f816573a88da1caf7 2023-02-11 03:55:38,251 DEBUG [StoreOpener-ea4735f297efbbfcc4cbc0bd3d6e42e0-1] regionserver.HStore(539): loaded hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/ea4735f297efbbfcc4cbc0bd3d6e42e0/family2/testCompactionWithHFileLink=ea4735f297efbbfcc4cbc0bd3d6e42e0-a77823d8bc254eb5b1130f536fa90187 2023-02-11 03:55:38,251 INFO [StoreOpener-ea4735f297efbbfcc4cbc0bd3d6e42e0-1] regionserver.HStore(310): Store=ea4735f297efbbfcc4cbc0bd3d6e42e0/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 03:55:38,252 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/ea4735f297efbbfcc4cbc0bd3d6e42e0 2023-02-11 03:55:38,253 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/ea4735f297efbbfcc4cbc0bd3d6e42e0 2023-02-11 03:55:38,254 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testCompactionWithHFileLink descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-11 03:55:38,255 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for ea4735f297efbbfcc4cbc0bd3d6e42e0 2023-02-11 03:55:38,256 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened ea4735f297efbbfcc4cbc0bd3d6e42e0; next sequenceid=32; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=73245086, jitterRate=0.09143683314323425}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-11 03:55:38,256 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for ea4735f297efbbfcc4cbc0bd3d6e42e0: Running coprocessor pre-open hook at 1676087738189Writing region info on filesystem at 1676087738189Initializing all the Stores at 1676087738189Instantiating store for column family {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1676087738189Instantiating store for column family {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1676087738190 (+1 ms)Cleaning up temporary data from old regions at 1676087738253 (+63 ms)Cleaning up detritus from prior splits at 1676087738254 (+1 ms)Running coprocessor post-open hooks at 1676087738256 (+2 ms)Region opened successfully at 1676087738256 2023-02-11 03:55:38,257 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testCompactionWithHFileLink,1,1676087711806.ea4735f297efbbfcc4cbc0bd3d6e42e0., pid=103, masterSystemTime=1676087738035 2023-02-11 03:55:38,257 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=3), splitQueue=0 2023-02-11 03:55:38,257 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=4), splitQueue=0 2023-02-11 03:55:38,258 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testCompactionWithHFileLink,1,1676087711806.ea4735f297efbbfcc4cbc0bd3d6e42e0. 2023-02-11 03:55:38,258 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testCompactionWithHFileLink,1,1676087711806.ea4735f297efbbfcc4cbc0bd3d6e42e0. 2023-02-11 03:55:38,258 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=99 updating hbase:meta row=ea4735f297efbbfcc4cbc0bd3d6e42e0, regionState=OPEN, openSeqNum=32, regionLocation=jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:55:38,259 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testCompactionWithHFileLink,1,1676087711806.ea4735f297efbbfcc4cbc0bd3d6e42e0.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1676087738258"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676087738258"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676087738258"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676087738258"}]},"ts":"1676087738258"} 2023-02-11 03:55:38,261 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=103, resume processing ppid=99 2023-02-11 03:55:38,261 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=103, ppid=99, state=SUCCESS; OpenRegionProcedure ea4735f297efbbfcc4cbc0bd3d6e42e0, server=jenkins-hbase4.apache.org,37011,1676087618428 in 375 msec 2023-02-11 03:55:38,262 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=99, resume processing ppid=97 2023-02-11 03:55:38,262 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=99, ppid=97, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=ea4735f297efbbfcc4cbc0bd3d6e42e0, ASSIGN in 532 msec 2023-02-11 03:55:38,263 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testCompactionWithHFileLink","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676087738263"}]},"ts":"1676087738263"} 2023-02-11 03:55:38,264 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=testCompactionWithHFileLink, state=ENABLED in hbase:meta 2023-02-11 03:55:38,266 INFO [PEWorker-4] procedure.EnableTableProcedure(310): Table 'testCompactionWithHFileLink' was successfully enabled. 2023-02-11 03:55:38,267 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=97, state=SUCCESS; EnableTableProcedure table=testCompactionWithHFileLink in 549 msec 2023-02-11 03:55:38,327 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39921] master.MasterRpcServices(1227): Checking to see if procedure is done pid=97 2023-02-11 03:55:38,327 INFO [Listener at localhost/46205] client.HBaseAdmin$TableFuture(3541): Operation: ENABLE, Table Name: default:testCompactionWithHFileLink, procId: 97 completed 2023-02-11 03:55:38,339 DEBUG [Listener at localhost/46205] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-02-11 03:55:38,345 DEBUG [Listener at localhost/46205] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-02-11 03:55:38,354 DEBUG [Listener at localhost/46205] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-02-11 03:55:38,355 DEBUG [Listener at localhost/46205] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-02-11 03:55:38,360 INFO [Listener at localhost/46205] compactions.PartitionedMobCompactor(142): is allFiles: false 2023-02-11 03:55:38,371 INFO [Listener at localhost/46205] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 12 selected files, and 0 irrelevant files table 'testCompactionWithHFileLink' and column 'family1' 2023-02-11 03:55:38,371 INFO [Listener at localhost/46205] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testCompactionWithHFileLink' column='family1' 2023-02-11 03:55:38,371 DEBUG [Listener at localhost/46205] zookeeper.ReadOnlyZKClient(139): Connect 0x32f4cb3b to 127.0.0.1:53383 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-11 03:55:38,376 DEBUG [Listener at localhost/46205] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@27841654, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-11 03:55:38,376 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c4ca4238a0b923820dcc509a6f75849b20230211 for table='testCompactionWithHFileLink' and column='family1' 2023-02-11 03:55:38,376 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20230211 for table='testCompactionWithHFileLink' and column='family1' 2023-02-11 03:55:38,376 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c81e728d9d4c2f636f067f89cc14862c20230211 for table='testCompactionWithHFileLink' and column='family1' 2023-02-11 03:55:38,867 INFO [MobFileCompactionChore-pool-1] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230211c0e3e5eea10848b598ff35f39a5e3593 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230211c0e3e5eea10848b598ff35f39a5e3593 2023-02-11 03:55:38,868 INFO [MobFileCompactionChore-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230211d2ce3b22c3c947ca9efa1b5c431f333b to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230211d2ce3b22c3c947ca9efa1b5c431f333b 2023-02-11 03:55:38,868 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20230211 table='testCompactionWithHFileLink' column='family1' 2023-02-11 03:55:38,869 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20230211 table='testCompactionWithHFileLink' column='family1' 2023-02-11 03:55:38,888 DEBUG [MobFileCompactionChore-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-11 03:55:38,893 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:60488, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-11 03:55:38,900 DEBUG [MobFileCompactionChore-pool-1] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-02-11 03:55:38,903 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:48674, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-02-11 03:55:38,916 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20230211/family1/c320ca21b2fe45c0bd851f86898ab61d first=Optional[2\x00\x00\x00\x00] last=Optional[2\x00\x00\x00\x07] 2023-02-11 03:55:38,916 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20230211/family1/798ad7d6083344f5a80fc1f5bec6be14 first=Optional[0\x00\x00\x00\x00] last=Optional[0\x00\x00\x00\x07] 2023-02-11 03:55:38,918 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testCompactionWithHFileLink,,1676087711806.186d87f306418fb25d295d824c1272fe., hostname=jenkins-hbase4.apache.org,37011,1676087618428, seqNum=32 for row with hfile group [{family1,hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20230211/family1/798ad7d6083344f5a80fc1f5bec6be14}] 2023-02-11 03:55:38,918 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testCompactionWithHFileLink,2,1676087711806.621cd59809a16fd910172a9890d26263., hostname=jenkins-hbase4.apache.org,37011,1676087618428, seqNum=32 for row 2 with hfile group [{family1,hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20230211/family1/c320ca21b2fe45c0bd851f86898ab61d}] 2023-02-11 03:55:38,925 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.HStore(763): Validating hfile at hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20230211/family1/798ad7d6083344f5a80fc1f5bec6be14 for inclusion in 186d87f306418fb25d295d824c1272fe/family1 2023-02-11 03:55:38,928 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37011] regionserver.HStore(763): Validating hfile at hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20230211/family1/c320ca21b2fe45c0bd851f86898ab61d for inclusion in 621cd59809a16fd910172a9890d26263/family1 2023-02-11 03:55:38,931 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37011] regionserver.HStore(775): HFile bounds: first=2\x00\x00\x00\x00 last=2\x00\x00\x00\x07 2023-02-11 03:55:38,931 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.HStore(775): HFile bounds: first=0\x00\x00\x00\x00 last=0\x00\x00\x00\x07 2023-02-11 03:55:38,931 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37011] regionserver.HStore(777): Region bounds: first=2 last= 2023-02-11 03:55:38,931 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.HStore(777): Region bounds: first= last=1 2023-02-11 03:55:38,932 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.HRegion(2445): Flush status journal for 186d87f306418fb25d295d824c1272fe: 2023-02-11 03:55:38,932 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37011] regionserver.HRegion(2445): Flush status journal for 621cd59809a16fd910172a9890d26263: 2023-02-11 03:55:38,933 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37011] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20230211/family1/c320ca21b2fe45c0bd851f86898ab61d to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/staging/jenkins__testCompactionWithHFileLink__3v6f3b1tp36ppis376ktqumf4vstr713bi1olcniu0q4e1vb27lg9bjhd5lorjjm/family1/c320ca21b2fe45c0bd851f86898ab61d 2023-02-11 03:55:38,933 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20230211/family1/798ad7d6083344f5a80fc1f5bec6be14 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/staging/jenkins__testCompactionWithHFileLink__nlng2cijvkou6o3pstlvgrit6ipa2kq8ao52ev1tlaugsmbf19u7064p6f3sh27n/family1/798ad7d6083344f5a80fc1f5bec6be14 2023-02-11 03:55:38,936 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37011] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/staging/jenkins__testCompactionWithHFileLink__3v6f3b1tp36ppis376ktqumf4vstr713bi1olcniu0q4e1vb27lg9bjhd5lorjjm/family1/c320ca21b2fe45c0bd851f86898ab61d as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/621cd59809a16fd910172a9890d26263/family1/18fdf0220f974ef58acc6933c297ad67_SeqId_34_ 2023-02-11 03:55:38,936 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/staging/jenkins__testCompactionWithHFileLink__nlng2cijvkou6o3pstlvgrit6ipa2kq8ao52ev1tlaugsmbf19u7064p6f3sh27n/family1/798ad7d6083344f5a80fc1f5bec6be14 as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/186d87f306418fb25d295d824c1272fe/family1/221b00ff8ccf4edf907fbed4feb7d725_SeqId_34_ 2023-02-11 03:55:38,937 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37011] regionserver.HStore(849): Loaded HFile hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/staging/jenkins__testCompactionWithHFileLink__3v6f3b1tp36ppis376ktqumf4vstr713bi1olcniu0q4e1vb27lg9bjhd5lorjjm/family1/c320ca21b2fe45c0bd851f86898ab61d into 621cd59809a16fd910172a9890d26263/family1 as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/621cd59809a16fd910172a9890d26263/family1/18fdf0220f974ef58acc6933c297ad67_SeqId_34_ - updating store file list. 2023-02-11 03:55:38,937 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.HStore(849): Loaded HFile hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/staging/jenkins__testCompactionWithHFileLink__nlng2cijvkou6o3pstlvgrit6ipa2kq8ao52ev1tlaugsmbf19u7064p6f3sh27n/family1/798ad7d6083344f5a80fc1f5bec6be14 into 186d87f306418fb25d295d824c1272fe/family1 as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/186d87f306418fb25d295d824c1272fe/family1/221b00ff8ccf4edf907fbed4feb7d725_SeqId_34_ - updating store file list. 2023-02-11 03:55:38,941 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.HStore(882): Loaded HFile hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/186d87f306418fb25d295d824c1272fe/family1/221b00ff8ccf4edf907fbed4feb7d725_SeqId_34_ into 186d87f306418fb25d295d824c1272fe/family1 2023-02-11 03:55:38,941 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.HStore(855): Successfully loaded hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/staging/jenkins__testCompactionWithHFileLink__nlng2cijvkou6o3pstlvgrit6ipa2kq8ao52ev1tlaugsmbf19u7064p6f3sh27n/family1/798ad7d6083344f5a80fc1f5bec6be14 into 186d87f306418fb25d295d824c1272fe/family1 (new location: hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/186d87f306418fb25d295d824c1272fe/family1/221b00ff8ccf4edf907fbed4feb7d725_SeqId_34_) 2023-02-11 03:55:38,941 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37011] regionserver.HStore(882): Loaded HFile hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/621cd59809a16fd910172a9890d26263/family1/18fdf0220f974ef58acc6933c297ad67_SeqId_34_ into 621cd59809a16fd910172a9890d26263/family1 2023-02-11 03:55:38,941 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37011] regionserver.HStore(855): Successfully loaded hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/staging/jenkins__testCompactionWithHFileLink__3v6f3b1tp36ppis376ktqumf4vstr713bi1olcniu0q4e1vb27lg9bjhd5lorjjm/family1/c320ca21b2fe45c0bd851f86898ab61d into 621cd59809a16fd910172a9890d26263/family1 (new location: hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/621cd59809a16fd910172a9890d26263/family1/18fdf0220f974ef58acc6933c297ad67_SeqId_34_) 2023-02-11 03:55:38,943 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/staging/jenkins__testCompactionWithHFileLink__nlng2cijvkou6o3pstlvgrit6ipa2kq8ao52ev1tlaugsmbf19u7064p6f3sh27n/family1/798ad7d6083344f5a80fc1f5bec6be14 2023-02-11 03:55:38,948 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37011] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/staging/jenkins__testCompactionWithHFileLink__3v6f3b1tp36ppis376ktqumf4vstr713bi1olcniu0q4e1vb27lg9bjhd5lorjjm/family1/c320ca21b2fe45c0bd851f86898ab61d 2023-02-11 03:55:38,948 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37011] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-02-11 03:55:38,950 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20230211 table='testCompactionWithHFileLink' column='family1' 2023-02-11 03:55:38,950 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-11 03:55:38,952 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-d41d8cd98f00b204e9800998ecf8427e20230211101da7200e8b4cf48af3f9e6b626325b to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-d41d8cd98f00b204e9800998ecf8427e20230211101da7200e8b4cf48af3f9e6b626325b 2023-02-11 03:55:38,952 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37011] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 5 files of size 29606 starting at candidate #0 after considering 6 permutations with 6 in ratio 2023-02-11 03:55:38,952 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37011] regionserver.HStore(1912): 621cd59809a16fd910172a9890d26263/family1 is initiating minor compaction (all files) 2023-02-11 03:55:38,952 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37011] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@356192d9; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=5), splitQueue=0 2023-02-11 03:55:38,953 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-d41d8cd98f00b204e9800998ecf8427e20230211e4a3d035f9844b869e82ec0923db93f1 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-d41d8cd98f00b204e9800998ecf8427e20230211e4a3d035f9844b869e82ec0923db93f1 2023-02-11 03:55:38,954 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-d41d8cd98f00b204e9800998ecf8427e20230211e861d00e5a114d6b996d9db1875d1dc1 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-d41d8cd98f00b204e9800998ecf8427e20230211e861d00e5a114d6b996d9db1875d1dc1 2023-02-11 03:55:38,955 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20230211 table='testCompactionWithHFileLink' column='family1' 2023-02-11 03:55:38,956 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-11 03:55:38,956 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-d41d8cd98f00b204e9800998ecf8427e20230211efe856603053418f95a0d3b45cd7d38d to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-d41d8cd98f00b204e9800998ecf8427e20230211efe856603053418f95a0d3b45cd7d38d 2023-02-11 03:55:38,956 INFO [MobFileCompactionChore-pool-0] mob.MobUtils(508): Table testCompactionWithHFileLink 4 expired mob files are deleted 2023-02-11 03:55:38,956 DEBUG [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20230211' 2023-02-11 03:55:38,957 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20230211 for table='testCompactionWithHFileLink' and column='family1' 2023-02-11 03:55:38,958 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c81e728d9d4c2f636f067f89cc14862c20230211464ca8637b744ea09bffcf8850a4d558 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c81e728d9d4c2f636f067f89cc14862c20230211464ca8637b744ea09bffcf8850a4d558 2023-02-11 03:55:38,959 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c81e728d9d4c2f636f067f89cc14862c2023021148e3d369a1484149bf86dd1e2169b387 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c81e728d9d4c2f636f067f89cc14862c2023021148e3d369a1484149bf86dd1e2169b387 2023-02-11 03:55:38,960 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c81e728d9d4c2f636f067f89cc14862c202302115097863fecd842c6a4c0b957f23811ff to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c81e728d9d4c2f636f067f89cc14862c202302115097863fecd842c6a4c0b957f23811ff 2023-02-11 03:55:38,962 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c81e728d9d4c2f636f067f89cc14862c202302116f5da94e2cf44f26952495b89148e481 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c81e728d9d4c2f636f067f89cc14862c202302116f5da94e2cf44f26952495b89148e481 2023-02-11 03:55:38,962 INFO [MobFileCompactionChore-pool-1] mob.MobUtils(508): Table testCompactionWithHFileLink 4 expired mob files are deleted 2023-02-11 03:55:38,962 DEBUG [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20230211' 2023-02-11 03:55:38,962 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=c81e728d9d4c2f636f067f89cc14862c20230211 for table='testCompactionWithHFileLink' and column='family1' 2023-02-11 03:55:39,263 INFO [MobFileCompactionChore-pool-2] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230211e301869d4ef0453eb67d575ddde6f251 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230211e301869d4ef0453eb67d575ddde6f251 2023-02-11 03:55:39,264 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c4ca4238a0b923820dcc509a6f75849b20230211 table='testCompactionWithHFileLink' column='family1' 2023-02-11 03:55:39,302 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20230211/family1/8d58cb1cfa57461c9a4ea83da97e4316 first=Optional[1\x00\x00\x00\x00] last=Optional[1\x00\x00\x00\x07] 2023-02-11 03:55:39,303 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testCompactionWithHFileLink,1,1676087711806.ea4735f297efbbfcc4cbc0bd3d6e42e0., hostname=jenkins-hbase4.apache.org,37011,1676087618428, seqNum=32 for row 1 with hfile group [{family1,hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20230211/family1/8d58cb1cfa57461c9a4ea83da97e4316}] 2023-02-11 03:55:39,305 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37011] regionserver.HStore(763): Validating hfile at hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20230211/family1/8d58cb1cfa57461c9a4ea83da97e4316 for inclusion in ea4735f297efbbfcc4cbc0bd3d6e42e0/family1 2023-02-11 03:55:39,307 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37011] regionserver.HStore(775): HFile bounds: first=1\x00\x00\x00\x00 last=1\x00\x00\x00\x07 2023-02-11 03:55:39,307 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37011] regionserver.HStore(777): Region bounds: first=1 last=2 2023-02-11 03:55:39,308 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37011] regionserver.HRegion(2445): Flush status journal for ea4735f297efbbfcc4cbc0bd3d6e42e0: 2023-02-11 03:55:39,309 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37011] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20230211/family1/8d58cb1cfa57461c9a4ea83da97e4316 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/staging/jenkins__testCompactionWithHFileLink__j2rtcd8fgchsuvc83ovehenjvsitnk2uolpt3rb10s1835gfjrq7clgj0dte7r9q/family1/8d58cb1cfa57461c9a4ea83da97e4316 2023-02-11 03:55:39,311 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37011] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/staging/jenkins__testCompactionWithHFileLink__j2rtcd8fgchsuvc83ovehenjvsitnk2uolpt3rb10s1835gfjrq7clgj0dte7r9q/family1/8d58cb1cfa57461c9a4ea83da97e4316 as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/ea4735f297efbbfcc4cbc0bd3d6e42e0/family1/3d99b5d4d0b249f2ab25aa17eeaf8d8f_SeqId_34_ 2023-02-11 03:55:39,311 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37011] regionserver.HStore(849): Loaded HFile hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/staging/jenkins__testCompactionWithHFileLink__j2rtcd8fgchsuvc83ovehenjvsitnk2uolpt3rb10s1835gfjrq7clgj0dte7r9q/family1/8d58cb1cfa57461c9a4ea83da97e4316 into ea4735f297efbbfcc4cbc0bd3d6e42e0/family1 as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/ea4735f297efbbfcc4cbc0bd3d6e42e0/family1/3d99b5d4d0b249f2ab25aa17eeaf8d8f_SeqId_34_ - updating store file list. 2023-02-11 03:55:39,314 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37011] regionserver.HStore(882): Loaded HFile hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/ea4735f297efbbfcc4cbc0bd3d6e42e0/family1/3d99b5d4d0b249f2ab25aa17eeaf8d8f_SeqId_34_ into ea4735f297efbbfcc4cbc0bd3d6e42e0/family1 2023-02-11 03:55:39,314 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37011] regionserver.HStore(855): Successfully loaded hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/staging/jenkins__testCompactionWithHFileLink__j2rtcd8fgchsuvc83ovehenjvsitnk2uolpt3rb10s1835gfjrq7clgj0dte7r9q/family1/8d58cb1cfa57461c9a4ea83da97e4316 into ea4735f297efbbfcc4cbc0bd3d6e42e0/family1 (new location: hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/ea4735f297efbbfcc4cbc0bd3d6e42e0/family1/3d99b5d4d0b249f2ab25aa17eeaf8d8f_SeqId_34_) 2023-02-11 03:55:39,314 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37011] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/staging/jenkins__testCompactionWithHFileLink__j2rtcd8fgchsuvc83ovehenjvsitnk2uolpt3rb10s1835gfjrq7clgj0dte7r9q/family1/8d58cb1cfa57461c9a4ea83da97e4316 2023-02-11 03:55:39,314 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37011] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-02-11 03:55:39,316 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37011] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 5 files of size 29606 starting at candidate #0 after considering 6 permutations with 6 in ratio 2023-02-11 03:55:39,316 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37011] regionserver.HStore(1912): ea4735f297efbbfcc4cbc0bd3d6e42e0/family1 is initiating minor compaction (all files) 2023-02-11 03:55:39,316 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37011] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@d8720b; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=6), splitQueue=0 2023-02-11 03:55:39,318 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c4ca4238a0b923820dcc509a6f75849b20230211 table='testCompactionWithHFileLink' column='family1' 2023-02-11 03:55:39,319 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(360): Archiving compacted files. 2023-02-11 03:55:39,320 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c4ca4238a0b923820dcc509a6f75849b2023021101c3e7c91909459eaa090b3ee73d4c03 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c4ca4238a0b923820dcc509a6f75849b2023021101c3e7c91909459eaa090b3ee73d4c03 2023-02-11 03:55:39,321 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c4ca4238a0b923820dcc509a6f75849b2023021143a646d95a904a22a34aeef0a1d76a19 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c4ca4238a0b923820dcc509a6f75849b2023021143a646d95a904a22a34aeef0a1d76a19 2023-02-11 03:55:39,322 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c4ca4238a0b923820dcc509a6f75849b2023021169ada67b5fa9484db7e2de93e45014c6 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c4ca4238a0b923820dcc509a6f75849b2023021169ada67b5fa9484db7e2de93e45014c6 2023-02-11 03:55:39,322 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c4ca4238a0b923820dcc509a6f75849b20230211bbcb00e25dbb444c96d85d72f4a0a09a to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c4ca4238a0b923820dcc509a6f75849b20230211bbcb00e25dbb444c96d85d72f4a0a09a 2023-02-11 03:55:39,322 INFO [MobFileCompactionChore-pool-2] mob.MobUtils(508): Table testCompactionWithHFileLink 4 expired mob files are deleted 2023-02-11 03:55:39,323 DEBUG [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20230211' 2023-02-11 03:55:39,323 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=c4ca4238a0b923820dcc509a6f75849b20230211 for table='testCompactionWithHFileLink' and column='family1' 2023-02-11 03:55:39,323 INFO [Listener at localhost/46205] compactions.PartitionedMobCompactor(354): After compaction, there are 3 mob files. table='testCompactionWithHFileLink' column='family1' 2023-02-11 03:55:39,326 DEBUG [Listener at localhost/46205] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-02-11 03:55:39,328 DEBUG [Listener at localhost/46205] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-02-11 03:55:39,332 DEBUG [Listener at localhost/46205] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-02-11 03:55:39,333 DEBUG [Listener at localhost/46205] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-02-11 03:55:39,341 DEBUG [Listener at localhost/46205] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-02-11 03:55:39,342 DEBUG [Listener at localhost/46205] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-02-11 03:55:39,360 INFO [Listener at localhost/46205] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testCompactionWithHFileLink Thread=285 (was 235) - Thread LEAK? -, OpenFileDescriptor=766 (was 589) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=111 (was 156), ProcessCount=168 (was 168), AvailableMemoryMB=2988 (was 3092) 2023-02-11 03:55:39,373 INFO [Listener at localhost/46205] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMinorCompaction Thread=285, OpenFileDescriptor=766, MaxFileDescriptor=60000, SystemLoadAverage=111, ProcessCount=168, AvailableMemoryMB=2987 2023-02-11 03:55:39,378 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] master.HMaster$15(2997): Client=jenkins//172.31.14.131 creating {NAME => 'ns'} 2023-02-11 03:55:39,380 DEBUG [RS:0;jenkins-hbase4:37011-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/186d87f306418fb25d295d824c1272fe/.tmp/family2/6dac8e26a79c4985b4c80b8f96fced1a as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/186d87f306418fb25d295d824c1272fe/family2/6dac8e26a79c4985b4c80b8f96fced1a 2023-02-11 03:55:39,380 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] procedure2.ProcedureExecutor(1029): Stored pid=104, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=ns 2023-02-11 03:55:39,382 DEBUG [RS:0;jenkins-hbase4:37011-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/186d87f306418fb25d295d824c1272fe/.tmp/family1/b9462a612cac4e4da54a3076e49f81f2 as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/186d87f306418fb25d295d824c1272fe/family1/b9462a612cac4e4da54a3076e49f81f2 2023-02-11 03:55:39,387 INFO [RS:0;jenkins-hbase4:37011-longCompactions-0] regionserver.HStore(1652): Completed compaction of 4 (all) file(s) in 186d87f306418fb25d295d824c1272fe/family2 of 186d87f306418fb25d295d824c1272fe into 6dac8e26a79c4985b4c80b8f96fced1a(size=6.4 K), total size for store is 6.4 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-02-11 03:55:39,387 DEBUG [RS:0;jenkins-hbase4:37011-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for 186d87f306418fb25d295d824c1272fe: 2023-02-11 03:55:39,387 INFO [RS:0;jenkins-hbase4:37011-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testCompactionWithHFileLink,,1676087711806.186d87f306418fb25d295d824c1272fe., storeName=186d87f306418fb25d295d824c1272fe/family2, priority=12, startTime=1676087738099; duration=1sec 2023-02-11 03:55:39,387 DEBUG [RS:0;jenkins-hbase4:37011-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=6), splitQueue=0 2023-02-11 03:55:39,387 INFO [RS:0;jenkins-hbase4:37011-longCompactions-0] regionserver.HRegion(2258): Starting compaction of 621cd59809a16fd910172a9890d26263/family1 in testCompactionWithHFileLink,2,1676087711806.621cd59809a16fd910172a9890d26263. 2023-02-11 03:55:39,387 INFO [RS:0;jenkins-hbase4:37011-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/621cd59809a16fd910172a9890d26263/family1/testCompactionWithHFileLink=621cd59809a16fd910172a9890d26263-a54c49bae3b34318920fc4197db7ae61, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/621cd59809a16fd910172a9890d26263/family1/testCompactionWithHFileLink=621cd59809a16fd910172a9890d26263-051a49ec8101444f8e462520d66fa021, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/621cd59809a16fd910172a9890d26263/family1/testCompactionWithHFileLink=621cd59809a16fd910172a9890d26263-3cb4dc3b66114935967cd14111320066, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/621cd59809a16fd910172a9890d26263/family1/testCompactionWithHFileLink=621cd59809a16fd910172a9890d26263-42ea1ea361834322b07cb7fa95ee893f, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/621cd59809a16fd910172a9890d26263/family1/18fdf0220f974ef58acc6933c297ad67_SeqId_34_] into tmpdir=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/621cd59809a16fd910172a9890d26263/.tmp, totalSize=28.9 K 2023-02-11 03:55:39,388 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] master.MasterRpcServices(1227): Checking to see if procedure is done pid=104 2023-02-11 03:55:39,389 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-02-11 03:55:39,393 DEBUG [RS:0;jenkins-hbase4:37011-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=621cd59809a16fd910172a9890d26263-a54c49bae3b34318920fc4197db7ae61, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1676087712922 2023-02-11 03:55:39,394 INFO [RS:0;jenkins-hbase4:37011-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 4 (all) file(s) in 186d87f306418fb25d295d824c1272fe/family1 of 186d87f306418fb25d295d824c1272fe into b9462a612cac4e4da54a3076e49f81f2(size=7.6 K), total size for store is 14.9 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-02-11 03:55:39,394 DEBUG [RS:0;jenkins-hbase4:37011-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 186d87f306418fb25d295d824c1272fe: 2023-02-11 03:55:39,394 INFO [RS:0;jenkins-hbase4:37011-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testCompactionWithHFileLink,,1676087711806.186d87f306418fb25d295d824c1272fe., storeName=186d87f306418fb25d295d824c1272fe/family1, priority=12, startTime=1676087738099; duration=1sec 2023-02-11 03:55:39,394 DEBUG [RS:0;jenkins-hbase4:37011-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=5), splitQueue=0 2023-02-11 03:55:39,394 INFO [RS:0;jenkins-hbase4:37011-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of ea4735f297efbbfcc4cbc0bd3d6e42e0/family1 in testCompactionWithHFileLink,1,1676087711806.ea4735f297efbbfcc4cbc0bd3d6e42e0. 2023-02-11 03:55:39,394 INFO [RS:0;jenkins-hbase4:37011-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/ea4735f297efbbfcc4cbc0bd3d6e42e0/family1/testCompactionWithHFileLink=ea4735f297efbbfcc4cbc0bd3d6e42e0-92743cf8525b4b5d877b358e975a3bec, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/ea4735f297efbbfcc4cbc0bd3d6e42e0/family1/testCompactionWithHFileLink=ea4735f297efbbfcc4cbc0bd3d6e42e0-0a642f51d6d94a79b4c5663961dee5c0, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/ea4735f297efbbfcc4cbc0bd3d6e42e0/family1/testCompactionWithHFileLink=ea4735f297efbbfcc4cbc0bd3d6e42e0-c504159ab71244858e1768f8eb16927b, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/ea4735f297efbbfcc4cbc0bd3d6e42e0/family1/testCompactionWithHFileLink=ea4735f297efbbfcc4cbc0bd3d6e42e0-3088ce87ec264e38859de9f824a4ab99, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/ea4735f297efbbfcc4cbc0bd3d6e42e0/family1/3d99b5d4d0b249f2ab25aa17eeaf8d8f_SeqId_34_] into tmpdir=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/ea4735f297efbbfcc4cbc0bd3d6e42e0/.tmp, totalSize=28.9 K 2023-02-11 03:55:39,395 DEBUG [RS:0;jenkins-hbase4:37011-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=621cd59809a16fd910172a9890d26263-051a49ec8101444f8e462520d66fa021, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1676087714701 2023-02-11 03:55:39,395 DEBUG [RS:0;jenkins-hbase4:37011-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=ea4735f297efbbfcc4cbc0bd3d6e42e0-92743cf8525b4b5d877b358e975a3bec, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1676087712921 2023-02-11 03:55:39,396 DEBUG [RS:0;jenkins-hbase4:37011-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=621cd59809a16fd910172a9890d26263-3cb4dc3b66114935967cd14111320066, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1676087716494 2023-02-11 03:55:39,396 DEBUG [RS:0;jenkins-hbase4:37011-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=ea4735f297efbbfcc4cbc0bd3d6e42e0-0a642f51d6d94a79b4c5663961dee5c0, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1676087714701 2023-02-11 03:55:39,396 DEBUG [RS:0;jenkins-hbase4:37011-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=621cd59809a16fd910172a9890d26263-42ea1ea361834322b07cb7fa95ee893f, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1676087718286 2023-02-11 03:55:39,397 DEBUG [RS:0;jenkins-hbase4:37011-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=ea4735f297efbbfcc4cbc0bd3d6e42e0-c504159ab71244858e1768f8eb16927b, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1676087716494 2023-02-11 03:55:39,397 DEBUG [RS:0;jenkins-hbase4:37011-longCompactions-0] compactions.Compactor(207): Compacting 18fdf0220f974ef58acc6933c297ad67_SeqId_34_, keycount=16, bloomtype=ROW, size=7.3 K, encoding=NONE, compression=NONE, seqNum=35, earliestPutTs=1676087712922 2023-02-11 03:55:39,398 DEBUG [RS:0;jenkins-hbase4:37011-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=ea4735f297efbbfcc4cbc0bd3d6e42e0-3088ce87ec264e38859de9f824a4ab99, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1676087718286 2023-02-11 03:55:39,399 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=104, state=SUCCESS; CreateNamespaceProcedure, namespace=ns in 17 msec 2023-02-11 03:55:39,399 DEBUG [RS:0;jenkins-hbase4:37011-shortCompactions-0] compactions.Compactor(207): Compacting 3d99b5d4d0b249f2ab25aa17eeaf8d8f_SeqId_34_, keycount=16, bloomtype=ROW, size=7.3 K, encoding=NONE, compression=NONE, seqNum=35, earliestPutTs=1676087712921 2023-02-11 03:55:39,417 INFO [RS:0;jenkins-hbase4:37011-longCompactions-0] throttle.PressureAwareThroughputController(145): 621cd59809a16fd910172a9890d26263#family1#compaction#115 average throughput is 0.47 MB/second, slept 0 time(s) and total slept time is 0 ms. 1 active operations remaining, total limit is 61.54 MB/second 2023-02-11 03:55:39,418 INFO [RS:0;jenkins-hbase4:37011-shortCompactions-0] throttle.PressureAwareThroughputController(145): ea4735f297efbbfcc4cbc0bd3d6e42e0#family1#compaction#116 average throughput is 1.17 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 61.54 MB/second 2023-02-11 03:55:39,451 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/186d87f306418fb25d295d824c1272fe/family1/testCompactionWithHFileLink=186d87f306418fb25d295d824c1272fe-2e6dcfa9c78543fcb2273f9d67b88e36, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/186d87f306418fb25d295d824c1272fe/family1/testCompactionWithHFileLink=186d87f306418fb25d295d824c1272fe-4e9479861f4540a6b8a67dee5f4b8c2c, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/186d87f306418fb25d295d824c1272fe/family1/testCompactionWithHFileLink=186d87f306418fb25d295d824c1272fe-55bf25ed3b544fe9969ac6430989cefc, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/186d87f306418fb25d295d824c1272fe/family1/testCompactionWithHFileLink=186d87f306418fb25d295d824c1272fe-1363e2a6dd1f49e0a9d550f758d2bcac] to archive 2023-02-11 03:55:39,452 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-11 03:55:39,454 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/186d87f306418fb25d295d824c1272fe/family1/testCompactionWithHFileLink=186d87f306418fb25d295d824c1272fe-2e6dcfa9c78543fcb2273f9d67b88e36 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testCompactionWithHFileLink/186d87f306418fb25d295d824c1272fe/family1/testCompactionWithHFileLink=186d87f306418fb25d295d824c1272fe-2e6dcfa9c78543fcb2273f9d67b88e36 2023-02-11 03:55:39,456 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/186d87f306418fb25d295d824c1272fe/family1/testCompactionWithHFileLink=186d87f306418fb25d295d824c1272fe-4e9479861f4540a6b8a67dee5f4b8c2c to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testCompactionWithHFileLink/186d87f306418fb25d295d824c1272fe/family1/testCompactionWithHFileLink=186d87f306418fb25d295d824c1272fe-4e9479861f4540a6b8a67dee5f4b8c2c 2023-02-11 03:55:39,459 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/186d87f306418fb25d295d824c1272fe/family1/testCompactionWithHFileLink=186d87f306418fb25d295d824c1272fe-55bf25ed3b544fe9969ac6430989cefc to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testCompactionWithHFileLink/186d87f306418fb25d295d824c1272fe/family1/testCompactionWithHFileLink=186d87f306418fb25d295d824c1272fe-55bf25ed3b544fe9969ac6430989cefc 2023-02-11 03:55:39,460 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/186d87f306418fb25d295d824c1272fe/family1/testCompactionWithHFileLink=186d87f306418fb25d295d824c1272fe-1363e2a6dd1f49e0a9d550f758d2bcac to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testCompactionWithHFileLink/186d87f306418fb25d295d824c1272fe/family1/testCompactionWithHFileLink=186d87f306418fb25d295d824c1272fe-1363e2a6dd1f49e0a9d550f758d2bcac 2023-02-11 03:55:39,462 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/186d87f306418fb25d295d824c1272fe/family2/testCompactionWithHFileLink=186d87f306418fb25d295d824c1272fe-48173b765b124663a97b2f43b90e8a88, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/186d87f306418fb25d295d824c1272fe/family2/testCompactionWithHFileLink=186d87f306418fb25d295d824c1272fe-039e3e0c12c742f29c5ea86a66e98c44, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/186d87f306418fb25d295d824c1272fe/family2/testCompactionWithHFileLink=186d87f306418fb25d295d824c1272fe-14bfe13c870f426d986ef125637ce8f4, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/186d87f306418fb25d295d824c1272fe/family2/testCompactionWithHFileLink=186d87f306418fb25d295d824c1272fe-203bb908b48a4908b252cec8e0b7c5e7] to archive 2023-02-11 03:55:39,463 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-11 03:55:39,465 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/186d87f306418fb25d295d824c1272fe/family2/testCompactionWithHFileLink=186d87f306418fb25d295d824c1272fe-48173b765b124663a97b2f43b90e8a88 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testCompactionWithHFileLink/186d87f306418fb25d295d824c1272fe/family2/testCompactionWithHFileLink=186d87f306418fb25d295d824c1272fe-48173b765b124663a97b2f43b90e8a88 2023-02-11 03:55:39,466 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/186d87f306418fb25d295d824c1272fe/family2/testCompactionWithHFileLink=186d87f306418fb25d295d824c1272fe-039e3e0c12c742f29c5ea86a66e98c44 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testCompactionWithHFileLink/186d87f306418fb25d295d824c1272fe/family2/testCompactionWithHFileLink=186d87f306418fb25d295d824c1272fe-039e3e0c12c742f29c5ea86a66e98c44 2023-02-11 03:55:39,467 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/186d87f306418fb25d295d824c1272fe/family2/testCompactionWithHFileLink=186d87f306418fb25d295d824c1272fe-14bfe13c870f426d986ef125637ce8f4 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testCompactionWithHFileLink/186d87f306418fb25d295d824c1272fe/family2/testCompactionWithHFileLink=186d87f306418fb25d295d824c1272fe-14bfe13c870f426d986ef125637ce8f4 2023-02-11 03:55:39,468 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/186d87f306418fb25d295d824c1272fe/family2/testCompactionWithHFileLink=186d87f306418fb25d295d824c1272fe-203bb908b48a4908b252cec8e0b7c5e7 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testCompactionWithHFileLink/186d87f306418fb25d295d824c1272fe/family2/testCompactionWithHFileLink=186d87f306418fb25d295d824c1272fe-203bb908b48a4908b252cec8e0b7c5e7 2023-02-11 03:55:39,489 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] master.MasterRpcServices(1227): Checking to see if procedure is done pid=104 2023-02-11 03:55:39,490 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] master.HMaster$4(2102): Client=jenkins//172.31.14.131 create 'ns:testMinorCompaction', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} 2023-02-11 03:55:39,491 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] procedure2.ProcedureExecutor(1029): Stored pid=105, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=ns:testMinorCompaction 2023-02-11 03:55:39,492 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=105, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=ns:testMinorCompaction execute state=CREATE_TABLE_PRE_OPERATION 2023-02-11 03:55:39,493 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] master.MasterRpcServices(697): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "ns" qualifier: "testMinorCompaction" procId is: 105 2023-02-11 03:55:39,493 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=105, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=ns:testMinorCompaction execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-11 03:55:39,493 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] master.MasterRpcServices(1227): Checking to see if procedure is done pid=105 2023-02-11 03:55:39,495 DEBUG [HFileArchiver-16] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/.tmp/data/ns/testMinorCompaction/6938f3b95bbc15aff8fb4a9d24d9aaf5 2023-02-11 03:55:39,495 DEBUG [HFileArchiver-17] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/.tmp/data/ns/testMinorCompaction/3fa91eb317d63eaff7b1021985f3ad00 2023-02-11 03:55:39,495 DEBUG [HFileArchiver-18] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/.tmp/data/ns/testMinorCompaction/08cc9e8540da3c04f9eb90b5ad80782a 2023-02-11 03:55:39,495 DEBUG [HFileArchiver-16] backup.HFileArchiver(153): Directory hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/.tmp/data/ns/testMinorCompaction/6938f3b95bbc15aff8fb4a9d24d9aaf5 empty. 2023-02-11 03:55:39,495 DEBUG [HFileArchiver-18] backup.HFileArchiver(153): Directory hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/.tmp/data/ns/testMinorCompaction/08cc9e8540da3c04f9eb90b5ad80782a empty. 2023-02-11 03:55:39,495 DEBUG [HFileArchiver-17] backup.HFileArchiver(153): Directory hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/.tmp/data/ns/testMinorCompaction/3fa91eb317d63eaff7b1021985f3ad00 empty. 2023-02-11 03:55:39,496 DEBUG [HFileArchiver-16] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/.tmp/data/ns/testMinorCompaction/6938f3b95bbc15aff8fb4a9d24d9aaf5 2023-02-11 03:55:39,496 DEBUG [HFileArchiver-18] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/.tmp/data/ns/testMinorCompaction/08cc9e8540da3c04f9eb90b5ad80782a 2023-02-11 03:55:39,496 DEBUG [HFileArchiver-17] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/.tmp/data/ns/testMinorCompaction/3fa91eb317d63eaff7b1021985f3ad00 2023-02-11 03:55:39,496 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived ns:testMinorCompaction regions 2023-02-11 03:55:39,594 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] master.MasterRpcServices(1227): Checking to see if procedure is done pid=105 2023-02-11 03:55:39,795 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] master.MasterRpcServices(1227): Checking to see if procedure is done pid=105 2023-02-11 03:55:39,906 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/.tmp/data/ns/testMinorCompaction/.tabledesc/.tableinfo.0000000001 2023-02-11 03:55:39,907 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(7671): creating {ENCODED => 6938f3b95bbc15aff8fb4a9d24d9aaf5, NAME => 'ns:testMinorCompaction,,1676087739490.6938f3b95bbc15aff8fb4a9d24d9aaf5.', STARTKEY => '', ENDKEY => '1'}, tableDescriptor='ns:testMinorCompaction', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/.tmp 2023-02-11 03:55:39,908 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(7671): creating {ENCODED => 08cc9e8540da3c04f9eb90b5ad80782a, NAME => 'ns:testMinorCompaction,2,1676087739490.08cc9e8540da3c04f9eb90b5ad80782a.', STARTKEY => '2', ENDKEY => ''}, tableDescriptor='ns:testMinorCompaction', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/.tmp 2023-02-11 03:55:39,908 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(7671): creating {ENCODED => 3fa91eb317d63eaff7b1021985f3ad00, NAME => 'ns:testMinorCompaction,1,1676087739490.3fa91eb317d63eaff7b1021985f3ad00.', STARTKEY => '1', ENDKEY => '2'}, tableDescriptor='ns:testMinorCompaction', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/.tmp 2023-02-11 03:55:40,096 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] master.MasterRpcServices(1227): Checking to see if procedure is done pid=105 2023-02-11 03:55:40,314 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(865): Instantiated ns:testMinorCompaction,2,1676087739490.08cc9e8540da3c04f9eb90b5ad80782a.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-11 03:55:40,314 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(865): Instantiated ns:testMinorCompaction,,1676087739490.6938f3b95bbc15aff8fb4a9d24d9aaf5.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-11 03:55:40,314 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(1603): Closing 08cc9e8540da3c04f9eb90b5ad80782a, disabling compactions & flushes 2023-02-11 03:55:40,314 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(1603): Closing 6938f3b95bbc15aff8fb4a9d24d9aaf5, disabling compactions & flushes 2023-02-11 03:55:40,314 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(865): Instantiated ns:testMinorCompaction,1,1676087739490.3fa91eb317d63eaff7b1021985f3ad00.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-11 03:55:40,314 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(1625): Closing region ns:testMinorCompaction,,1676087739490.6938f3b95bbc15aff8fb4a9d24d9aaf5. 2023-02-11 03:55:40,314 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(1603): Closing 3fa91eb317d63eaff7b1021985f3ad00, disabling compactions & flushes 2023-02-11 03:55:40,314 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on ns:testMinorCompaction,,1676087739490.6938f3b95bbc15aff8fb4a9d24d9aaf5. 2023-02-11 03:55:40,314 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(1625): Closing region ns:testMinorCompaction,1,1676087739490.3fa91eb317d63eaff7b1021985f3ad00. 2023-02-11 03:55:40,314 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(1713): Acquired close lock on ns:testMinorCompaction,,1676087739490.6938f3b95bbc15aff8fb4a9d24d9aaf5. after waiting 0 ms 2023-02-11 03:55:40,314 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(1723): Updates disabled for region ns:testMinorCompaction,,1676087739490.6938f3b95bbc15aff8fb4a9d24d9aaf5. 2023-02-11 03:55:40,314 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(1625): Closing region ns:testMinorCompaction,2,1676087739490.08cc9e8540da3c04f9eb90b5ad80782a. 2023-02-11 03:55:40,314 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(1837): Closed ns:testMinorCompaction,,1676087739490.6938f3b95bbc15aff8fb4a9d24d9aaf5. 2023-02-11 03:55:40,314 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(1646): Waiting without time limit for close lock on ns:testMinorCompaction,1,1676087739490.3fa91eb317d63eaff7b1021985f3ad00. 2023-02-11 03:55:40,314 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(1557): Region close journal for 6938f3b95bbc15aff8fb4a9d24d9aaf5: Waiting for close lock at 1676087740314Disabling compacts and flushes for region at 1676087740314Disabling writes for close at 1676087740314Writing region close event to WAL at 1676087740314Closed at 1676087740314 2023-02-11 03:55:40,314 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(1646): Waiting without time limit for close lock on ns:testMinorCompaction,2,1676087739490.08cc9e8540da3c04f9eb90b5ad80782a. 2023-02-11 03:55:40,314 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(1713): Acquired close lock on ns:testMinorCompaction,1,1676087739490.3fa91eb317d63eaff7b1021985f3ad00. after waiting 0 ms 2023-02-11 03:55:40,314 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(1713): Acquired close lock on ns:testMinorCompaction,2,1676087739490.08cc9e8540da3c04f9eb90b5ad80782a. after waiting 0 ms 2023-02-11 03:55:40,315 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(1723): Updates disabled for region ns:testMinorCompaction,2,1676087739490.08cc9e8540da3c04f9eb90b5ad80782a. 2023-02-11 03:55:40,314 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(1723): Updates disabled for region ns:testMinorCompaction,1,1676087739490.3fa91eb317d63eaff7b1021985f3ad00. 2023-02-11 03:55:40,315 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(1837): Closed ns:testMinorCompaction,1,1676087739490.3fa91eb317d63eaff7b1021985f3ad00. 2023-02-11 03:55:40,315 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(1837): Closed ns:testMinorCompaction,2,1676087739490.08cc9e8540da3c04f9eb90b5ad80782a. 2023-02-11 03:55:40,315 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(1557): Region close journal for 3fa91eb317d63eaff7b1021985f3ad00: Waiting for close lock at 1676087740314Disabling compacts and flushes for region at 1676087740314Disabling writes for close at 1676087740314Writing region close event to WAL at 1676087740315 (+1 ms)Closed at 1676087740315 2023-02-11 03:55:40,315 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(1557): Region close journal for 08cc9e8540da3c04f9eb90b5ad80782a: Waiting for close lock at 1676087740314Disabling compacts and flushes for region at 1676087740314Disabling writes for close at 1676087740314Writing region close event to WAL at 1676087740315 (+1 ms)Closed at 1676087740315 2023-02-11 03:55:40,317 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=105, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=ns:testMinorCompaction execute state=CREATE_TABLE_ADD_TO_META 2023-02-11 03:55:40,318 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"ns:testMinorCompaction,,1676087739490.6938f3b95bbc15aff8fb4a9d24d9aaf5.","families":{"info":[{"qualifier":"regioninfo","vlen":49,"tag":[],"timestamp":"1676087740318"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676087740318"}]},"ts":"1676087740318"} 2023-02-11 03:55:40,318 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"ns:testMinorCompaction,1,1676087739490.3fa91eb317d63eaff7b1021985f3ad00.","families":{"info":[{"qualifier":"regioninfo","vlen":50,"tag":[],"timestamp":"1676087740318"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676087740318"}]},"ts":"1676087740318"} 2023-02-11 03:55:40,318 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"ns:testMinorCompaction,2,1676087739490.08cc9e8540da3c04f9eb90b5ad80782a.","families":{"info":[{"qualifier":"regioninfo","vlen":49,"tag":[],"timestamp":"1676087740318"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676087740318"}]},"ts":"1676087740318"} 2023-02-11 03:55:40,320 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 3 regions to meta. 2023-02-11 03:55:40,321 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=105, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=ns:testMinorCompaction execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-11 03:55:40,321 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"ns:testMinorCompaction","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676087740321"}]},"ts":"1676087740321"} 2023-02-11 03:55:40,322 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=ns:testMinorCompaction, state=ENABLING in hbase:meta 2023-02-11 03:55:40,325 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=106, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=ns:testMinorCompaction, region=6938f3b95bbc15aff8fb4a9d24d9aaf5, ASSIGN}, {pid=107, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=ns:testMinorCompaction, region=3fa91eb317d63eaff7b1021985f3ad00, ASSIGN}, {pid=108, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=ns:testMinorCompaction, region=08cc9e8540da3c04f9eb90b5ad80782a, ASSIGN}] 2023-02-11 03:55:40,327 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=108, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=ns:testMinorCompaction, region=08cc9e8540da3c04f9eb90b5ad80782a, ASSIGN 2023-02-11 03:55:40,327 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=107, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=ns:testMinorCompaction, region=3fa91eb317d63eaff7b1021985f3ad00, ASSIGN 2023-02-11 03:55:40,327 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=106, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=ns:testMinorCompaction, region=6938f3b95bbc15aff8fb4a9d24d9aaf5, ASSIGN 2023-02-11 03:55:40,327 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=108, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=ns:testMinorCompaction, region=08cc9e8540da3c04f9eb90b5ad80782a, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,37011,1676087618428; forceNewPlan=false, retain=false 2023-02-11 03:55:40,327 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=107, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=ns:testMinorCompaction, region=3fa91eb317d63eaff7b1021985f3ad00, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,37011,1676087618428; forceNewPlan=false, retain=false 2023-02-11 03:55:40,327 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=106, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=ns:testMinorCompaction, region=6938f3b95bbc15aff8fb4a9d24d9aaf5, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,37011,1676087618428; forceNewPlan=false, retain=false 2023-02-11 03:55:40,339 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-11 03:55:40,480 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=106 updating hbase:meta row=6938f3b95bbc15aff8fb4a9d24d9aaf5, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:55:40,480 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=107 updating hbase:meta row=3fa91eb317d63eaff7b1021985f3ad00, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:55:40,480 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=108 updating hbase:meta row=08cc9e8540da3c04f9eb90b5ad80782a, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:55:40,480 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"ns:testMinorCompaction,1,1676087739490.3fa91eb317d63eaff7b1021985f3ad00.","families":{"info":[{"qualifier":"regioninfo","vlen":50,"tag":[],"timestamp":"1676087740480"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676087740480"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676087740480"}]},"ts":"1676087740480"} 2023-02-11 03:55:40,480 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"ns:testMinorCompaction,,1676087739490.6938f3b95bbc15aff8fb4a9d24d9aaf5.","families":{"info":[{"qualifier":"regioninfo","vlen":49,"tag":[],"timestamp":"1676087740480"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676087740480"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676087740480"}]},"ts":"1676087740480"} 2023-02-11 03:55:40,480 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"ns:testMinorCompaction,2,1676087739490.08cc9e8540da3c04f9eb90b5ad80782a.","families":{"info":[{"qualifier":"regioninfo","vlen":49,"tag":[],"timestamp":"1676087740480"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676087740480"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676087740480"}]},"ts":"1676087740480"} 2023-02-11 03:55:40,481 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=109, ppid=107, state=RUNNABLE; OpenRegionProcedure 3fa91eb317d63eaff7b1021985f3ad00, server=jenkins-hbase4.apache.org,37011,1676087618428}] 2023-02-11 03:55:40,482 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=110, ppid=106, state=RUNNABLE; OpenRegionProcedure 6938f3b95bbc15aff8fb4a9d24d9aaf5, server=jenkins-hbase4.apache.org,37011,1676087618428}] 2023-02-11 03:55:40,483 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=111, ppid=108, state=RUNNABLE; OpenRegionProcedure 08cc9e8540da3c04f9eb90b5ad80782a, server=jenkins-hbase4.apache.org,37011,1676087618428}] 2023-02-11 03:55:40,597 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] master.MasterRpcServices(1227): Checking to see if procedure is done pid=105 2023-02-11 03:55:40,635 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open ns:testMinorCompaction,,1676087739490.6938f3b95bbc15aff8fb4a9d24d9aaf5. 2023-02-11 03:55:40,635 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 6938f3b95bbc15aff8fb4a9d24d9aaf5, NAME => 'ns:testMinorCompaction,,1676087739490.6938f3b95bbc15aff8fb4a9d24d9aaf5.', STARTKEY => '', ENDKEY => '1'} 2023-02-11 03:55:40,636 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMinorCompaction 6938f3b95bbc15aff8fb4a9d24d9aaf5 2023-02-11 03:55:40,636 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated ns:testMinorCompaction,,1676087739490.6938f3b95bbc15aff8fb4a9d24d9aaf5.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-11 03:55:40,636 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 6938f3b95bbc15aff8fb4a9d24d9aaf5 2023-02-11 03:55:40,636 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 6938f3b95bbc15aff8fb4a9d24d9aaf5 2023-02-11 03:55:40,637 INFO [StoreOpener-6938f3b95bbc15aff8fb4a9d24d9aaf5-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 6938f3b95bbc15aff8fb4a9d24d9aaf5 2023-02-11 03:55:40,638 DEBUG [StoreOpener-6938f3b95bbc15aff8fb4a9d24d9aaf5-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/6938f3b95bbc15aff8fb4a9d24d9aaf5/family1 2023-02-11 03:55:40,638 DEBUG [StoreOpener-6938f3b95bbc15aff8fb4a9d24d9aaf5-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/6938f3b95bbc15aff8fb4a9d24d9aaf5/family1 2023-02-11 03:55:40,639 INFO [StoreOpener-6938f3b95bbc15aff8fb4a9d24d9aaf5-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 6938f3b95bbc15aff8fb4a9d24d9aaf5 columnFamilyName family1 2023-02-11 03:55:40,640 INFO [StoreOpener-6938f3b95bbc15aff8fb4a9d24d9aaf5-1] regionserver.HStore(310): Store=6938f3b95bbc15aff8fb4a9d24d9aaf5/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 03:55:40,640 INFO [StoreOpener-6938f3b95bbc15aff8fb4a9d24d9aaf5-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 6938f3b95bbc15aff8fb4a9d24d9aaf5 2023-02-11 03:55:40,641 DEBUG [StoreOpener-6938f3b95bbc15aff8fb4a9d24d9aaf5-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/6938f3b95bbc15aff8fb4a9d24d9aaf5/family2 2023-02-11 03:55:40,641 DEBUG [StoreOpener-6938f3b95bbc15aff8fb4a9d24d9aaf5-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/6938f3b95bbc15aff8fb4a9d24d9aaf5/family2 2023-02-11 03:55:40,642 INFO [StoreOpener-6938f3b95bbc15aff8fb4a9d24d9aaf5-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 6938f3b95bbc15aff8fb4a9d24d9aaf5 columnFamilyName family2 2023-02-11 03:55:40,643 INFO [StoreOpener-6938f3b95bbc15aff8fb4a9d24d9aaf5-1] regionserver.HStore(310): Store=6938f3b95bbc15aff8fb4a9d24d9aaf5/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 03:55:40,644 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/6938f3b95bbc15aff8fb4a9d24d9aaf5 2023-02-11 03:55:40,644 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/6938f3b95bbc15aff8fb4a9d24d9aaf5 2023-02-11 03:55:40,647 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table ns:testMinorCompaction descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-11 03:55:40,647 DEBUG [RS:0;jenkins-hbase4:37011-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/621cd59809a16fd910172a9890d26263/.tmp/family1/b2aacde50c534760bbdb52dfb0595ca4 as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/621cd59809a16fd910172a9890d26263/family1/b2aacde50c534760bbdb52dfb0595ca4 2023-02-11 03:55:40,647 DEBUG [RS:0;jenkins-hbase4:37011-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/ea4735f297efbbfcc4cbc0bd3d6e42e0/.tmp/family1/3a2635c678b24772b2851e40879b1c7e as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/ea4735f297efbbfcc4cbc0bd3d6e42e0/family1/3a2635c678b24772b2851e40879b1c7e 2023-02-11 03:55:40,648 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 6938f3b95bbc15aff8fb4a9d24d9aaf5 2023-02-11 03:55:40,650 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/6938f3b95bbc15aff8fb4a9d24d9aaf5/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-11 03:55:40,650 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 6938f3b95bbc15aff8fb4a9d24d9aaf5; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=66652900, jitterRate=-0.006794393062591553}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-11 03:55:40,650 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 6938f3b95bbc15aff8fb4a9d24d9aaf5: Running coprocessor pre-open hook at 1676087740636Writing region info on filesystem at 1676087740636Initializing all the Stores at 1676087740637 (+1 ms)Instantiating store for column family {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1676087740637Instantiating store for column family {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1676087740637Cleaning up temporary data from old regions at 1676087740645 (+8 ms)Cleaning up detritus from prior splits at 1676087740646 (+1 ms)Running coprocessor post-open hooks at 1676087740650 (+4 ms)Region opened successfully at 1676087740650 2023-02-11 03:55:40,651 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for ns:testMinorCompaction,,1676087739490.6938f3b95bbc15aff8fb4a9d24d9aaf5., pid=110, masterSystemTime=1676087740633 2023-02-11 03:55:40,652 INFO [RS:0;jenkins-hbase4:37011-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 5 (all) file(s) in ea4735f297efbbfcc4cbc0bd3d6e42e0/family1 of ea4735f297efbbfcc4cbc0bd3d6e42e0 into 3a2635c678b24772b2851e40879b1c7e(size=7.6 K), total size for store is 7.6 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-02-11 03:55:40,652 INFO [RS:0;jenkins-hbase4:37011-longCompactions-0] regionserver.HStore(1652): Completed compaction of 5 (all) file(s) in 621cd59809a16fd910172a9890d26263/family1 of 621cd59809a16fd910172a9890d26263 into b2aacde50c534760bbdb52dfb0595ca4(size=7.6 K), total size for store is 7.6 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-02-11 03:55:40,652 DEBUG [RS:0;jenkins-hbase4:37011-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for ea4735f297efbbfcc4cbc0bd3d6e42e0: 2023-02-11 03:55:40,652 DEBUG [RS:0;jenkins-hbase4:37011-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for 621cd59809a16fd910172a9890d26263: 2023-02-11 03:55:40,652 INFO [RS:0;jenkins-hbase4:37011-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testCompactionWithHFileLink,1,1676087711806.ea4735f297efbbfcc4cbc0bd3d6e42e0., storeName=family1, fileCount=5, fileSize=28.9 K (5.4 K, 5.4 K, 5.4 K, 5.4 K, 7.3 K), priority=2, time=1676087739316; duration=1sec 2023-02-11 03:55:40,652 INFO [RS:0;jenkins-hbase4:37011-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testCompactionWithHFileLink,2,1676087711806.621cd59809a16fd910172a9890d26263., storeName=family1, fileCount=5, fileSize=28.9 K (5.4 K, 5.4 K, 5.4 K, 5.4 K, 7.3 K), priority=2, time=1676087738952; duration=1sec 2023-02-11 03:55:40,652 DEBUG [RS:0;jenkins-hbase4:37011-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=4), splitQueue=0 2023-02-11 03:55:40,652 DEBUG [RS:0;jenkins-hbase4:37011-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=3), splitQueue=0 2023-02-11 03:55:40,652 DEBUG [RS:0;jenkins-hbase4:37011-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 4 store files, 0 compacting, 4 eligible, 16 blocking 2023-02-11 03:55:40,652 DEBUG [RS:0;jenkins-hbase4:37011-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 4 store files, 0 compacting, 4 eligible, 16 blocking 2023-02-11 03:55:40,653 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for ns:testMinorCompaction,,1676087739490.6938f3b95bbc15aff8fb4a9d24d9aaf5. 2023-02-11 03:55:40,653 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened ns:testMinorCompaction,,1676087739490.6938f3b95bbc15aff8fb4a9d24d9aaf5. 2023-02-11 03:55:40,653 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open ns:testMinorCompaction,1,1676087739490.3fa91eb317d63eaff7b1021985f3ad00. 2023-02-11 03:55:40,653 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 3fa91eb317d63eaff7b1021985f3ad00, NAME => 'ns:testMinorCompaction,1,1676087739490.3fa91eb317d63eaff7b1021985f3ad00.', STARTKEY => '1', ENDKEY => '2'} 2023-02-11 03:55:40,653 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=106 updating hbase:meta row=6938f3b95bbc15aff8fb4a9d24d9aaf5, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:55:40,653 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMinorCompaction 3fa91eb317d63eaff7b1021985f3ad00 2023-02-11 03:55:40,653 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"ns:testMinorCompaction,,1676087739490.6938f3b95bbc15aff8fb4a9d24d9aaf5.","families":{"info":[{"qualifier":"regioninfo","vlen":49,"tag":[],"timestamp":"1676087740653"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676087740653"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676087740653"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676087740653"}]},"ts":"1676087740653"} 2023-02-11 03:55:40,653 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated ns:testMinorCompaction,1,1676087739490.3fa91eb317d63eaff7b1021985f3ad00.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-11 03:55:40,653 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 3fa91eb317d63eaff7b1021985f3ad00 2023-02-11 03:55:40,653 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 3fa91eb317d63eaff7b1021985f3ad00 2023-02-11 03:55:40,654 INFO [StoreOpener-3fa91eb317d63eaff7b1021985f3ad00-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 3fa91eb317d63eaff7b1021985f3ad00 2023-02-11 03:55:40,655 DEBUG [RS:0;jenkins-hbase4:37011-shortCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 4 files of size 20920 starting at candidate #0 after considering 3 permutations with 3 in ratio 2023-02-11 03:55:40,655 DEBUG [RS:0;jenkins-hbase4:37011-shortCompactions-0] regionserver.HStore(1912): 621cd59809a16fd910172a9890d26263/family2 is initiating minor compaction (all files) 2023-02-11 03:55:40,655 INFO [RS:0;jenkins-hbase4:37011-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 621cd59809a16fd910172a9890d26263/family2 in testCompactionWithHFileLink,2,1676087711806.621cd59809a16fd910172a9890d26263. 2023-02-11 03:55:40,655 DEBUG [RS:0;jenkins-hbase4:37011-longCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 4 files of size 20920 starting at candidate #0 after considering 3 permutations with 3 in ratio 2023-02-11 03:55:40,655 INFO [RS:0;jenkins-hbase4:37011-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/621cd59809a16fd910172a9890d26263/family2/testCompactionWithHFileLink=621cd59809a16fd910172a9890d26263-774f5f8b4d2e463f983d817bdedf591e, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/621cd59809a16fd910172a9890d26263/family2/testCompactionWithHFileLink=621cd59809a16fd910172a9890d26263-e000a979eed34f31ba1476e0d7c949da, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/621cd59809a16fd910172a9890d26263/family2/testCompactionWithHFileLink=621cd59809a16fd910172a9890d26263-87ccd53b473f47b6887f864a901e3b7c, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/621cd59809a16fd910172a9890d26263/family2/testCompactionWithHFileLink=621cd59809a16fd910172a9890d26263-32530f87de8648c0ab8a83636ca1711f] into tmpdir=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/621cd59809a16fd910172a9890d26263/.tmp, totalSize=20.4 K 2023-02-11 03:55:40,655 DEBUG [RS:0;jenkins-hbase4:37011-longCompactions-0] regionserver.HStore(1912): ea4735f297efbbfcc4cbc0bd3d6e42e0/family2 is initiating minor compaction (all files) 2023-02-11 03:55:40,655 INFO [RS:0;jenkins-hbase4:37011-longCompactions-0] regionserver.HRegion(2258): Starting compaction of ea4735f297efbbfcc4cbc0bd3d6e42e0/family2 in testCompactionWithHFileLink,1,1676087711806.ea4735f297efbbfcc4cbc0bd3d6e42e0. 2023-02-11 03:55:40,655 INFO [RS:0;jenkins-hbase4:37011-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/ea4735f297efbbfcc4cbc0bd3d6e42e0/family2/testCompactionWithHFileLink=ea4735f297efbbfcc4cbc0bd3d6e42e0-7141c179434b4a0f816573a88da1caf7, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/ea4735f297efbbfcc4cbc0bd3d6e42e0/family2/testCompactionWithHFileLink=ea4735f297efbbfcc4cbc0bd3d6e42e0-045314dc19574fe9979fbedc8e14acf8, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/ea4735f297efbbfcc4cbc0bd3d6e42e0/family2/testCompactionWithHFileLink=ea4735f297efbbfcc4cbc0bd3d6e42e0-a77823d8bc254eb5b1130f536fa90187, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/ea4735f297efbbfcc4cbc0bd3d6e42e0/family2/testCompactionWithHFileLink=ea4735f297efbbfcc4cbc0bd3d6e42e0-5b41b5f362864186bb9f20144e3bc4c5] into tmpdir=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/ea4735f297efbbfcc4cbc0bd3d6e42e0/.tmp, totalSize=20.4 K 2023-02-11 03:55:40,656 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=110, resume processing ppid=106 2023-02-11 03:55:40,656 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=110, ppid=106, state=SUCCESS; OpenRegionProcedure 6938f3b95bbc15aff8fb4a9d24d9aaf5, server=jenkins-hbase4.apache.org,37011,1676087618428 in 172 msec 2023-02-11 03:55:40,656 DEBUG [RS:0;jenkins-hbase4:37011-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=621cd59809a16fd910172a9890d26263-774f5f8b4d2e463f983d817bdedf591e, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1676087712922 2023-02-11 03:55:40,656 DEBUG [StoreOpener-3fa91eb317d63eaff7b1021985f3ad00-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/3fa91eb317d63eaff7b1021985f3ad00/family1 2023-02-11 03:55:40,657 DEBUG [StoreOpener-3fa91eb317d63eaff7b1021985f3ad00-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/3fa91eb317d63eaff7b1021985f3ad00/family1 2023-02-11 03:55:40,657 DEBUG [RS:0;jenkins-hbase4:37011-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=ea4735f297efbbfcc4cbc0bd3d6e42e0-7141c179434b4a0f816573a88da1caf7, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1676087712921 2023-02-11 03:55:40,657 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=106, ppid=105, state=SUCCESS; TransitRegionStateProcedure table=ns:testMinorCompaction, region=6938f3b95bbc15aff8fb4a9d24d9aaf5, ASSIGN in 331 msec 2023-02-11 03:55:40,657 INFO [StoreOpener-3fa91eb317d63eaff7b1021985f3ad00-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 3fa91eb317d63eaff7b1021985f3ad00 columnFamilyName family1 2023-02-11 03:55:40,657 DEBUG [RS:0;jenkins-hbase4:37011-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=621cd59809a16fd910172a9890d26263-e000a979eed34f31ba1476e0d7c949da, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1676087714701 2023-02-11 03:55:40,657 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/621cd59809a16fd910172a9890d26263/family1/testCompactionWithHFileLink=621cd59809a16fd910172a9890d26263-a54c49bae3b34318920fc4197db7ae61, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/621cd59809a16fd910172a9890d26263/family1/testCompactionWithHFileLink=621cd59809a16fd910172a9890d26263-051a49ec8101444f8e462520d66fa021, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/621cd59809a16fd910172a9890d26263/family1/testCompactionWithHFileLink=621cd59809a16fd910172a9890d26263-3cb4dc3b66114935967cd14111320066, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/621cd59809a16fd910172a9890d26263/family1/testCompactionWithHFileLink=621cd59809a16fd910172a9890d26263-42ea1ea361834322b07cb7fa95ee893f, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/621cd59809a16fd910172a9890d26263/family1/18fdf0220f974ef58acc6933c297ad67_SeqId_34_] to archive 2023-02-11 03:55:40,657 DEBUG [RS:0;jenkins-hbase4:37011-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=ea4735f297efbbfcc4cbc0bd3d6e42e0-045314dc19574fe9979fbedc8e14acf8, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1676087714701 2023-02-11 03:55:40,658 INFO [StoreOpener-3fa91eb317d63eaff7b1021985f3ad00-1] regionserver.HStore(310): Store=3fa91eb317d63eaff7b1021985f3ad00/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 03:55:40,658 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-11 03:55:40,658 DEBUG [RS:0;jenkins-hbase4:37011-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=621cd59809a16fd910172a9890d26263-87ccd53b473f47b6887f864a901e3b7c, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1676087716494 2023-02-11 03:55:40,658 DEBUG [RS:0;jenkins-hbase4:37011-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=ea4735f297efbbfcc4cbc0bd3d6e42e0-a77823d8bc254eb5b1130f536fa90187, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1676087716494 2023-02-11 03:55:40,658 INFO [StoreOpener-3fa91eb317d63eaff7b1021985f3ad00-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 3fa91eb317d63eaff7b1021985f3ad00 2023-02-11 03:55:40,659 DEBUG [RS:0;jenkins-hbase4:37011-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=621cd59809a16fd910172a9890d26263-32530f87de8648c0ab8a83636ca1711f, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1676087718286 2023-02-11 03:55:40,659 DEBUG [RS:0;jenkins-hbase4:37011-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=ea4735f297efbbfcc4cbc0bd3d6e42e0-5b41b5f362864186bb9f20144e3bc4c5, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1676087718286 2023-02-11 03:55:40,660 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/621cd59809a16fd910172a9890d26263/family1/testCompactionWithHFileLink=621cd59809a16fd910172a9890d26263-a54c49bae3b34318920fc4197db7ae61 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testCompactionWithHFileLink/621cd59809a16fd910172a9890d26263/family1/testCompactionWithHFileLink=621cd59809a16fd910172a9890d26263-a54c49bae3b34318920fc4197db7ae61 2023-02-11 03:55:40,660 DEBUG [StoreOpener-3fa91eb317d63eaff7b1021985f3ad00-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/3fa91eb317d63eaff7b1021985f3ad00/family2 2023-02-11 03:55:40,660 DEBUG [StoreOpener-3fa91eb317d63eaff7b1021985f3ad00-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/3fa91eb317d63eaff7b1021985f3ad00/family2 2023-02-11 03:55:40,660 INFO [StoreOpener-3fa91eb317d63eaff7b1021985f3ad00-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 3fa91eb317d63eaff7b1021985f3ad00 columnFamilyName family2 2023-02-11 03:55:40,661 INFO [StoreOpener-3fa91eb317d63eaff7b1021985f3ad00-1] regionserver.HStore(310): Store=3fa91eb317d63eaff7b1021985f3ad00/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 03:55:40,661 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/621cd59809a16fd910172a9890d26263/family1/testCompactionWithHFileLink=621cd59809a16fd910172a9890d26263-051a49ec8101444f8e462520d66fa021 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testCompactionWithHFileLink/621cd59809a16fd910172a9890d26263/family1/testCompactionWithHFileLink=621cd59809a16fd910172a9890d26263-051a49ec8101444f8e462520d66fa021 2023-02-11 03:55:40,663 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/3fa91eb317d63eaff7b1021985f3ad00 2023-02-11 03:55:40,663 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/621cd59809a16fd910172a9890d26263/family1/testCompactionWithHFileLink=621cd59809a16fd910172a9890d26263-3cb4dc3b66114935967cd14111320066 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testCompactionWithHFileLink/621cd59809a16fd910172a9890d26263/family1/testCompactionWithHFileLink=621cd59809a16fd910172a9890d26263-3cb4dc3b66114935967cd14111320066 2023-02-11 03:55:40,663 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/3fa91eb317d63eaff7b1021985f3ad00 2023-02-11 03:55:40,665 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/621cd59809a16fd910172a9890d26263/family1/testCompactionWithHFileLink=621cd59809a16fd910172a9890d26263-42ea1ea361834322b07cb7fa95ee893f to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testCompactionWithHFileLink/621cd59809a16fd910172a9890d26263/family1/testCompactionWithHFileLink=621cd59809a16fd910172a9890d26263-42ea1ea361834322b07cb7fa95ee893f 2023-02-11 03:55:40,666 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table ns:testMinorCompaction descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-11 03:55:40,667 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/621cd59809a16fd910172a9890d26263/family1/18fdf0220f974ef58acc6933c297ad67_SeqId_34_ to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testCompactionWithHFileLink/621cd59809a16fd910172a9890d26263/family1/18fdf0220f974ef58acc6933c297ad67_SeqId_34_ 2023-02-11 03:55:40,668 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 3fa91eb317d63eaff7b1021985f3ad00 2023-02-11 03:55:40,669 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/ea4735f297efbbfcc4cbc0bd3d6e42e0/family1/testCompactionWithHFileLink=ea4735f297efbbfcc4cbc0bd3d6e42e0-92743cf8525b4b5d877b358e975a3bec, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/ea4735f297efbbfcc4cbc0bd3d6e42e0/family1/testCompactionWithHFileLink=ea4735f297efbbfcc4cbc0bd3d6e42e0-0a642f51d6d94a79b4c5663961dee5c0, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/ea4735f297efbbfcc4cbc0bd3d6e42e0/family1/testCompactionWithHFileLink=ea4735f297efbbfcc4cbc0bd3d6e42e0-c504159ab71244858e1768f8eb16927b, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/ea4735f297efbbfcc4cbc0bd3d6e42e0/family1/testCompactionWithHFileLink=ea4735f297efbbfcc4cbc0bd3d6e42e0-3088ce87ec264e38859de9f824a4ab99, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/ea4735f297efbbfcc4cbc0bd3d6e42e0/family1/3d99b5d4d0b249f2ab25aa17eeaf8d8f_SeqId_34_] to archive 2023-02-11 03:55:40,670 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-11 03:55:40,670 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/3fa91eb317d63eaff7b1021985f3ad00/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-11 03:55:40,671 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 3fa91eb317d63eaff7b1021985f3ad00; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=58787379, jitterRate=-0.12399978935718536}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-11 03:55:40,671 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 3fa91eb317d63eaff7b1021985f3ad00: Running coprocessor pre-open hook at 1676087740653Writing region info on filesystem at 1676087740653Initializing all the Stores at 1676087740654 (+1 ms)Instantiating store for column family {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1676087740654Instantiating store for column family {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1676087740654Cleaning up temporary data from old regions at 1676087740664 (+10 ms)Cleaning up detritus from prior splits at 1676087740665 (+1 ms)Running coprocessor post-open hooks at 1676087740671 (+6 ms)Region opened successfully at 1676087740671 2023-02-11 03:55:40,672 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for ns:testMinorCompaction,1,1676087739490.3fa91eb317d63eaff7b1021985f3ad00., pid=109, masterSystemTime=1676087740633 2023-02-11 03:55:40,672 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/ea4735f297efbbfcc4cbc0bd3d6e42e0/family1/testCompactionWithHFileLink=ea4735f297efbbfcc4cbc0bd3d6e42e0-92743cf8525b4b5d877b358e975a3bec to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testCompactionWithHFileLink/ea4735f297efbbfcc4cbc0bd3d6e42e0/family1/testCompactionWithHFileLink=ea4735f297efbbfcc4cbc0bd3d6e42e0-92743cf8525b4b5d877b358e975a3bec 2023-02-11 03:55:40,673 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for ns:testMinorCompaction,1,1676087739490.3fa91eb317d63eaff7b1021985f3ad00. 2023-02-11 03:55:40,673 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened ns:testMinorCompaction,1,1676087739490.3fa91eb317d63eaff7b1021985f3ad00. 2023-02-11 03:55:40,674 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open ns:testMinorCompaction,2,1676087739490.08cc9e8540da3c04f9eb90b5ad80782a. 2023-02-11 03:55:40,674 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=107 updating hbase:meta row=3fa91eb317d63eaff7b1021985f3ad00, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:55:40,674 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 08cc9e8540da3c04f9eb90b5ad80782a, NAME => 'ns:testMinorCompaction,2,1676087739490.08cc9e8540da3c04f9eb90b5ad80782a.', STARTKEY => '2', ENDKEY => ''} 2023-02-11 03:55:40,674 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/ea4735f297efbbfcc4cbc0bd3d6e42e0/family1/testCompactionWithHFileLink=ea4735f297efbbfcc4cbc0bd3d6e42e0-0a642f51d6d94a79b4c5663961dee5c0 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testCompactionWithHFileLink/ea4735f297efbbfcc4cbc0bd3d6e42e0/family1/testCompactionWithHFileLink=ea4735f297efbbfcc4cbc0bd3d6e42e0-0a642f51d6d94a79b4c5663961dee5c0 2023-02-11 03:55:40,674 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"ns:testMinorCompaction,1,1676087739490.3fa91eb317d63eaff7b1021985f3ad00.","families":{"info":[{"qualifier":"regioninfo","vlen":50,"tag":[],"timestamp":"1676087740673"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676087740673"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676087740673"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676087740673"}]},"ts":"1676087740673"} 2023-02-11 03:55:40,674 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMinorCompaction 08cc9e8540da3c04f9eb90b5ad80782a 2023-02-11 03:55:40,674 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated ns:testMinorCompaction,2,1676087739490.08cc9e8540da3c04f9eb90b5ad80782a.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-11 03:55:40,674 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 08cc9e8540da3c04f9eb90b5ad80782a 2023-02-11 03:55:40,674 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 08cc9e8540da3c04f9eb90b5ad80782a 2023-02-11 03:55:40,675 INFO [StoreOpener-08cc9e8540da3c04f9eb90b5ad80782a-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 08cc9e8540da3c04f9eb90b5ad80782a 2023-02-11 03:55:40,676 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/ea4735f297efbbfcc4cbc0bd3d6e42e0/family1/testCompactionWithHFileLink=ea4735f297efbbfcc4cbc0bd3d6e42e0-c504159ab71244858e1768f8eb16927b to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testCompactionWithHFileLink/ea4735f297efbbfcc4cbc0bd3d6e42e0/family1/testCompactionWithHFileLink=ea4735f297efbbfcc4cbc0bd3d6e42e0-c504159ab71244858e1768f8eb16927b 2023-02-11 03:55:40,677 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=109, resume processing ppid=107 2023-02-11 03:55:40,677 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=109, ppid=107, state=SUCCESS; OpenRegionProcedure 3fa91eb317d63eaff7b1021985f3ad00, server=jenkins-hbase4.apache.org,37011,1676087618428 in 194 msec 2023-02-11 03:55:40,677 DEBUG [StoreOpener-08cc9e8540da3c04f9eb90b5ad80782a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/08cc9e8540da3c04f9eb90b5ad80782a/family1 2023-02-11 03:55:40,677 DEBUG [StoreOpener-08cc9e8540da3c04f9eb90b5ad80782a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/08cc9e8540da3c04f9eb90b5ad80782a/family1 2023-02-11 03:55:40,677 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/ea4735f297efbbfcc4cbc0bd3d6e42e0/family1/testCompactionWithHFileLink=ea4735f297efbbfcc4cbc0bd3d6e42e0-3088ce87ec264e38859de9f824a4ab99 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testCompactionWithHFileLink/ea4735f297efbbfcc4cbc0bd3d6e42e0/family1/testCompactionWithHFileLink=ea4735f297efbbfcc4cbc0bd3d6e42e0-3088ce87ec264e38859de9f824a4ab99 2023-02-11 03:55:40,678 INFO [StoreOpener-08cc9e8540da3c04f9eb90b5ad80782a-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 08cc9e8540da3c04f9eb90b5ad80782a columnFamilyName family1 2023-02-11 03:55:40,678 INFO [RS:0;jenkins-hbase4:37011-shortCompactions-0] throttle.PressureAwareThroughputController(145): 621cd59809a16fd910172a9890d26263#family2#compaction#117 average throughput is 0.39 MB/second, slept 0 time(s) and total slept time is 0 ms. 1 active operations remaining, total limit is 57.69 MB/second 2023-02-11 03:55:40,679 INFO [RS:0;jenkins-hbase4:37011-longCompactions-0] throttle.PressureAwareThroughputController(145): ea4735f297efbbfcc4cbc0bd3d6e42e0#family2#compaction#118 average throughput is 0.39 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 57.69 MB/second 2023-02-11 03:55:40,679 INFO [StoreOpener-08cc9e8540da3c04f9eb90b5ad80782a-1] regionserver.HStore(310): Store=08cc9e8540da3c04f9eb90b5ad80782a/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 03:55:40,679 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=107, ppid=105, state=SUCCESS; TransitRegionStateProcedure table=ns:testMinorCompaction, region=3fa91eb317d63eaff7b1021985f3ad00, ASSIGN in 352 msec 2023-02-11 03:55:40,679 INFO [StoreOpener-08cc9e8540da3c04f9eb90b5ad80782a-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 08cc9e8540da3c04f9eb90b5ad80782a 2023-02-11 03:55:40,679 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/ea4735f297efbbfcc4cbc0bd3d6e42e0/family1/3d99b5d4d0b249f2ab25aa17eeaf8d8f_SeqId_34_ to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testCompactionWithHFileLink/ea4735f297efbbfcc4cbc0bd3d6e42e0/family1/3d99b5d4d0b249f2ab25aa17eeaf8d8f_SeqId_34_ 2023-02-11 03:55:40,681 DEBUG [StoreOpener-08cc9e8540da3c04f9eb90b5ad80782a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/08cc9e8540da3c04f9eb90b5ad80782a/family2 2023-02-11 03:55:40,681 DEBUG [StoreOpener-08cc9e8540da3c04f9eb90b5ad80782a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/08cc9e8540da3c04f9eb90b5ad80782a/family2 2023-02-11 03:55:40,681 INFO [StoreOpener-08cc9e8540da3c04f9eb90b5ad80782a-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 08cc9e8540da3c04f9eb90b5ad80782a columnFamilyName family2 2023-02-11 03:55:40,682 INFO [StoreOpener-08cc9e8540da3c04f9eb90b5ad80782a-1] regionserver.HStore(310): Store=08cc9e8540da3c04f9eb90b5ad80782a/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 03:55:40,684 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/08cc9e8540da3c04f9eb90b5ad80782a 2023-02-11 03:55:40,685 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/08cc9e8540da3c04f9eb90b5ad80782a 2023-02-11 03:55:40,687 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table ns:testMinorCompaction descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-11 03:55:40,688 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 08cc9e8540da3c04f9eb90b5ad80782a 2023-02-11 03:55:40,690 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/08cc9e8540da3c04f9eb90b5ad80782a/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-11 03:55:40,690 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 08cc9e8540da3c04f9eb90b5ad80782a; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=61461875, jitterRate=-0.08414669334888458}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-11 03:55:40,690 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 08cc9e8540da3c04f9eb90b5ad80782a: Running coprocessor pre-open hook at 1676087740674Writing region info on filesystem at 1676087740674Initializing all the Stores at 1676087740675 (+1 ms)Instantiating store for column family {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1676087740675Instantiating store for column family {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1676087740675Cleaning up temporary data from old regions at 1676087740686 (+11 ms)Cleaning up detritus from prior splits at 1676087740686Running coprocessor post-open hooks at 1676087740690 (+4 ms)Region opened successfully at 1676087740690 2023-02-11 03:55:40,691 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for ns:testMinorCompaction,2,1676087739490.08cc9e8540da3c04f9eb90b5ad80782a., pid=111, masterSystemTime=1676087740633 2023-02-11 03:55:40,692 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for ns:testMinorCompaction,2,1676087739490.08cc9e8540da3c04f9eb90b5ad80782a. 2023-02-11 03:55:40,692 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened ns:testMinorCompaction,2,1676087739490.08cc9e8540da3c04f9eb90b5ad80782a. 2023-02-11 03:55:40,693 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=108 updating hbase:meta row=08cc9e8540da3c04f9eb90b5ad80782a, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:55:40,693 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"ns:testMinorCompaction,2,1676087739490.08cc9e8540da3c04f9eb90b5ad80782a.","families":{"info":[{"qualifier":"regioninfo","vlen":49,"tag":[],"timestamp":"1676087740693"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676087740693"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676087740693"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676087740693"}]},"ts":"1676087740693"} 2023-02-11 03:55:40,695 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=111, resume processing ppid=108 2023-02-11 03:55:40,695 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=111, ppid=108, state=SUCCESS; OpenRegionProcedure 08cc9e8540da3c04f9eb90b5ad80782a, server=jenkins-hbase4.apache.org,37011,1676087618428 in 211 msec 2023-02-11 03:55:40,698 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=108, resume processing ppid=105 2023-02-11 03:55:40,698 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=108, ppid=105, state=SUCCESS; TransitRegionStateProcedure table=ns:testMinorCompaction, region=08cc9e8540da3c04f9eb90b5ad80782a, ASSIGN in 370 msec 2023-02-11 03:55:40,698 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=105, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=ns:testMinorCompaction execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-11 03:55:40,698 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"ns:testMinorCompaction","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676087740698"}]},"ts":"1676087740698"} 2023-02-11 03:55:40,700 INFO [PEWorker-2] hbase.MetaTableAccessor(1635): Updated tableName=ns:testMinorCompaction, state=ENABLED in hbase:meta 2023-02-11 03:55:40,702 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=105, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=ns:testMinorCompaction execute state=CREATE_TABLE_POST_OPERATION 2023-02-11 03:55:40,703 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=105, state=SUCCESS; CreateTableProcedure table=ns:testMinorCompaction in 1.2120 sec 2023-02-11 03:55:41,598 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] master.MasterRpcServices(1227): Checking to see if procedure is done pid=105 2023-02-11 03:55:41,598 INFO [Listener at localhost/46205] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: ns:testMinorCompaction, procId: 105 completed 2023-02-11 03:55:41,604 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37011] regionserver.HRegion(8801): writing data to region ns:testMinorCompaction,,1676087739490.6938f3b95bbc15aff8fb4a9d24d9aaf5. with WAL disabled. Data may be lost in the event of a crash. 2023-02-11 03:55:41,604 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37011] regionserver.HRegion(8801): writing data to region ns:testMinorCompaction,1,1676087739490.3fa91eb317d63eaff7b1021985f3ad00. with WAL disabled. Data may be lost in the event of a crash. 2023-02-11 03:55:41,605 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37011] regionserver.HRegion(8801): writing data to region ns:testMinorCompaction,2,1676087739490.08cc9e8540da3c04f9eb90b5ad80782a. with WAL disabled. Data may be lost in the event of a crash. 2023-02-11 03:55:41,607 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 03:55:41,608 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] procedure.ProcedureCoordinator(165): Submitting procedure ns:testMinorCompaction 2023-02-11 03:55:41,608 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'ns:testMinorCompaction' 2023-02-11 03:55:41,608 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:55:41,608 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'ns:testMinorCompaction' starting 'acquire' 2023-02-11 03:55:41,608 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'ns:testMinorCompaction', kicking off acquire phase on members. 2023-02-11 03:55:41,608 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-11 03:55:41,608 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-02-11 03:55:41,610 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:55:41,610 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:55:41,610 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:55:41,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:55:41,610 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:55:41,610 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 03:55:41,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-02-11 03:55:41,611 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-11 03:55:41,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 03:55:41,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-02-11 03:55:41,611 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for ns:testMinorCompaction 2023-02-11 03:55:41,611 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:ns:testMinorCompaction 2023-02-11 03:55:41,611 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'ns:testMinorCompaction' with timeout 60000ms 2023-02-11 03:55:41,611 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:55:41,611 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'ns:testMinorCompaction' starting 'acquire' stage 2023-02-11 03:55:41,613 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,,1676087739490.6938f3b95bbc15aff8fb4a9d24d9aaf5. 2023-02-11 03:55:41,613 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,,1676087739490.6938f3b95bbc15aff8fb4a9d24d9aaf5. started... 2023-02-11 03:55:41,613 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-02-11 03:55:41,613 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,1,1676087739490.3fa91eb317d63eaff7b1021985f3ad00. 2023-02-11 03:55:41,613 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,2,1676087739490.08cc9e8540da3c04f9eb90b5ad80782a. 2023-02-11 03:55:41,615 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 6938f3b95bbc15aff8fb4a9d24d9aaf5 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-02-11 03:55:41,613 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 03:55:41,615 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,2,1676087739490.08cc9e8540da3c04f9eb90b5ad80782a. started... 2023-02-11 03:55:41,615 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,1,1676087739490.3fa91eb317d63eaff7b1021985f3ad00. started... 2023-02-11 03:55:41,615 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-2] regionserver.HRegion(2744): Flushing 08cc9e8540da3c04f9eb90b5ad80782a 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-02-11 03:55:41,615 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-1] regionserver.HRegion(2744): Flushing 3fa91eb317d63eaff7b1021985f3ad00 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-02-11 03:55:41,901 DEBUG [RS:0;jenkins-hbase4:37011-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/621cd59809a16fd910172a9890d26263/.tmp/family2/6d5758a2c4e74e1ba04509f9d42b90e7 as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/621cd59809a16fd910172a9890d26263/family2/6d5758a2c4e74e1ba04509f9d42b90e7 2023-02-11 03:55:41,901 DEBUG [RS:0;jenkins-hbase4:37011-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/ea4735f297efbbfcc4cbc0bd3d6e42e0/.tmp/family2/22a8a08b75564994b4fb65ef7b8fe8c4 as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/ea4735f297efbbfcc4cbc0bd3d6e42e0/family2/22a8a08b75564994b4fb65ef7b8fe8c4 2023-02-11 03:55:41,905 INFO [RS:0;jenkins-hbase4:37011-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 4 (all) file(s) in 621cd59809a16fd910172a9890d26263/family2 of 621cd59809a16fd910172a9890d26263 into 6d5758a2c4e74e1ba04509f9d42b90e7(size=6.4 K), total size for store is 6.4 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-02-11 03:55:41,905 INFO [RS:0;jenkins-hbase4:37011-longCompactions-0] regionserver.HStore(1652): Completed compaction of 4 (all) file(s) in ea4735f297efbbfcc4cbc0bd3d6e42e0/family2 of ea4735f297efbbfcc4cbc0bd3d6e42e0 into 22a8a08b75564994b4fb65ef7b8fe8c4(size=6.4 K), total size for store is 6.4 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-02-11 03:55:41,905 DEBUG [RS:0;jenkins-hbase4:37011-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 621cd59809a16fd910172a9890d26263: 2023-02-11 03:55:41,905 DEBUG [RS:0;jenkins-hbase4:37011-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for ea4735f297efbbfcc4cbc0bd3d6e42e0: 2023-02-11 03:55:41,905 INFO [RS:0;jenkins-hbase4:37011-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testCompactionWithHFileLink,2,1676087711806.621cd59809a16fd910172a9890d26263., storeName=621cd59809a16fd910172a9890d26263/family2, priority=12, startTime=1676087738187; duration=1sec 2023-02-11 03:55:41,905 INFO [RS:0;jenkins-hbase4:37011-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testCompactionWithHFileLink,1,1676087711806.ea4735f297efbbfcc4cbc0bd3d6e42e0., storeName=ea4735f297efbbfcc4cbc0bd3d6e42e0/family2, priority=12, startTime=1676087738257; duration=1sec 2023-02-11 03:55:41,905 DEBUG [RS:0;jenkins-hbase4:37011-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=2), splitQueue=0 2023-02-11 03:55:41,905 DEBUG [RS:0;jenkins-hbase4:37011-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-11 03:55:41,905 DEBUG [RS:0;jenkins-hbase4:37011-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 1 store files, 0 compacting, 1 eligible, 16 blocking 2023-02-11 03:55:41,905 DEBUG [RS:0;jenkins-hbase4:37011-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 1 store files, 0 compacting, 1 eligible, 16 blocking 2023-02-11 03:55:41,906 DEBUG [RS:0;jenkins-hbase4:37011-shortCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 0 files of size 0 starting at candidate #-1 after considering 0 permutations with 0 in ratio 2023-02-11 03:55:41,906 DEBUG [RS:0;jenkins-hbase4:37011-shortCompactions-0] compactions.SortedCompactionPolicy(232): Not compacting files because we only have 0 files ready for compaction. Need 3 to initiate. 2023-02-11 03:55:41,906 DEBUG [RS:0;jenkins-hbase4:37011-longCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 0 files of size 0 starting at candidate #-1 after considering 0 permutations with 0 in ratio 2023-02-11 03:55:41,906 DEBUG [RS:0;jenkins-hbase4:37011-longCompactions-0] compactions.SortedCompactionPolicy(232): Not compacting files because we only have 0 files ready for compaction. Need 3 to initiate. 2023-02-11 03:55:41,906 DEBUG [RS:0;jenkins-hbase4:37011-shortCompactions-0] regionserver.CompactSplit(415): Not compacting testCompactionWithHFileLink,1,1676087711806.ea4735f297efbbfcc4cbc0bd3d6e42e0. because compaction request was cancelled 2023-02-11 03:55:41,906 DEBUG [RS:0;jenkins-hbase4:37011-longCompactions-0] regionserver.CompactSplit(415): Not compacting testCompactionWithHFileLink,2,1676087711806.621cd59809a16fd910172a9890d26263. because compaction request was cancelled 2023-02-11 03:55:41,950 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/621cd59809a16fd910172a9890d26263/family2/testCompactionWithHFileLink=621cd59809a16fd910172a9890d26263-774f5f8b4d2e463f983d817bdedf591e, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/621cd59809a16fd910172a9890d26263/family2/testCompactionWithHFileLink=621cd59809a16fd910172a9890d26263-e000a979eed34f31ba1476e0d7c949da, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/621cd59809a16fd910172a9890d26263/family2/testCompactionWithHFileLink=621cd59809a16fd910172a9890d26263-87ccd53b473f47b6887f864a901e3b7c, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/621cd59809a16fd910172a9890d26263/family2/testCompactionWithHFileLink=621cd59809a16fd910172a9890d26263-32530f87de8648c0ab8a83636ca1711f] to archive 2023-02-11 03:55:41,951 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-11 03:55:41,952 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/621cd59809a16fd910172a9890d26263/family2/testCompactionWithHFileLink=621cd59809a16fd910172a9890d26263-774f5f8b4d2e463f983d817bdedf591e to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testCompactionWithHFileLink/621cd59809a16fd910172a9890d26263/family2/testCompactionWithHFileLink=621cd59809a16fd910172a9890d26263-774f5f8b4d2e463f983d817bdedf591e 2023-02-11 03:55:41,953 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/621cd59809a16fd910172a9890d26263/family2/testCompactionWithHFileLink=621cd59809a16fd910172a9890d26263-e000a979eed34f31ba1476e0d7c949da to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testCompactionWithHFileLink/621cd59809a16fd910172a9890d26263/family2/testCompactionWithHFileLink=621cd59809a16fd910172a9890d26263-e000a979eed34f31ba1476e0d7c949da 2023-02-11 03:55:41,955 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/621cd59809a16fd910172a9890d26263/family2/testCompactionWithHFileLink=621cd59809a16fd910172a9890d26263-87ccd53b473f47b6887f864a901e3b7c to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testCompactionWithHFileLink/621cd59809a16fd910172a9890d26263/family2/testCompactionWithHFileLink=621cd59809a16fd910172a9890d26263-87ccd53b473f47b6887f864a901e3b7c 2023-02-11 03:55:41,955 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/621cd59809a16fd910172a9890d26263/family2/testCompactionWithHFileLink=621cd59809a16fd910172a9890d26263-32530f87de8648c0ab8a83636ca1711f to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testCompactionWithHFileLink/621cd59809a16fd910172a9890d26263/family2/testCompactionWithHFileLink=621cd59809a16fd910172a9890d26263-32530f87de8648c0ab8a83636ca1711f 2023-02-11 03:55:41,957 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/ea4735f297efbbfcc4cbc0bd3d6e42e0/family2/testCompactionWithHFileLink=ea4735f297efbbfcc4cbc0bd3d6e42e0-7141c179434b4a0f816573a88da1caf7, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/ea4735f297efbbfcc4cbc0bd3d6e42e0/family2/testCompactionWithHFileLink=ea4735f297efbbfcc4cbc0bd3d6e42e0-045314dc19574fe9979fbedc8e14acf8, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/ea4735f297efbbfcc4cbc0bd3d6e42e0/family2/testCompactionWithHFileLink=ea4735f297efbbfcc4cbc0bd3d6e42e0-a77823d8bc254eb5b1130f536fa90187, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/ea4735f297efbbfcc4cbc0bd3d6e42e0/family2/testCompactionWithHFileLink=ea4735f297efbbfcc4cbc0bd3d6e42e0-5b41b5f362864186bb9f20144e3bc4c5] to archive 2023-02-11 03:55:41,957 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-11 03:55:41,958 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/ea4735f297efbbfcc4cbc0bd3d6e42e0/family2/testCompactionWithHFileLink=ea4735f297efbbfcc4cbc0bd3d6e42e0-7141c179434b4a0f816573a88da1caf7 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testCompactionWithHFileLink/ea4735f297efbbfcc4cbc0bd3d6e42e0/family2/testCompactionWithHFileLink=ea4735f297efbbfcc4cbc0bd3d6e42e0-7141c179434b4a0f816573a88da1caf7 2023-02-11 03:55:41,960 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/ea4735f297efbbfcc4cbc0bd3d6e42e0/family2/testCompactionWithHFileLink=ea4735f297efbbfcc4cbc0bd3d6e42e0-045314dc19574fe9979fbedc8e14acf8 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testCompactionWithHFileLink/ea4735f297efbbfcc4cbc0bd3d6e42e0/family2/testCompactionWithHFileLink=ea4735f297efbbfcc4cbc0bd3d6e42e0-045314dc19574fe9979fbedc8e14acf8 2023-02-11 03:55:41,961 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/ea4735f297efbbfcc4cbc0bd3d6e42e0/family2/testCompactionWithHFileLink=ea4735f297efbbfcc4cbc0bd3d6e42e0-a77823d8bc254eb5b1130f536fa90187 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testCompactionWithHFileLink/ea4735f297efbbfcc4cbc0bd3d6e42e0/family2/testCompactionWithHFileLink=ea4735f297efbbfcc4cbc0bd3d6e42e0-a77823d8bc254eb5b1130f536fa90187 2023-02-11 03:55:41,962 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/ea4735f297efbbfcc4cbc0bd3d6e42e0/family2/testCompactionWithHFileLink=ea4735f297efbbfcc4cbc0bd3d6e42e0-5b41b5f362864186bb9f20144e3bc4c5 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testCompactionWithHFileLink/ea4735f297efbbfcc4cbc0bd3d6e42e0/family2/testCompactionWithHFileLink=ea4735f297efbbfcc4cbc0bd3d6e42e0-5b41b5f362864186bb9f20144e3bc4c5 2023-02-11 03:55:42,028 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202302118b2f27fc9704450fbba349a924ac5f5d to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c4ca4238a0b923820dcc509a6f75849b202302118b2f27fc9704450fbba349a924ac5f5d 2023-02-11 03:55:42,028 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c2023021170c6e59bf6274b85aecdb9321c195f05 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c81e728d9d4c2f636f067f89cc14862c2023021170c6e59bf6274b85aecdb9321c195f05 2023-02-11 03:55:42,028 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230211742c6d88e07649a28098ce0ba5e992e5 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/d41d8cd98f00b204e9800998ecf8427e20230211742c6d88e07649a28098ce0ba5e992e5 2023-02-11 03:55:42,034 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/3fa91eb317d63eaff7b1021985f3ad00/.tmp/family1/4da5a5dcbded4cb9ad85833159091832 2023-02-11 03:55:42,035 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/6938f3b95bbc15aff8fb4a9d24d9aaf5/.tmp/family1/cf56fda2084b4736b3389b67629c9b2a 2023-02-11 03:55:42,435 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/08cc9e8540da3c04f9eb90b5ad80782a/.tmp/family1/1965537a06594723af60377479103e74 2023-02-11 03:55:42,460 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230211a707e364d1ea49939f915079f4125f22 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c4ca4238a0b923820dcc509a6f75849b20230211a707e364d1ea49939f915079f4125f22 2023-02-11 03:55:42,464 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230211c22cfe25ef424948acc7291459412419 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/d41d8cd98f00b204e9800998ecf8427e20230211c22cfe25ef424948acc7291459412419 2023-02-11 03:55:42,468 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/6938f3b95bbc15aff8fb4a9d24d9aaf5/.tmp/family2/9ec3c577685c478c85883a8dfde7d662 2023-02-11 03:55:42,471 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/6938f3b95bbc15aff8fb4a9d24d9aaf5/.tmp/family1/cf56fda2084b4736b3389b67629c9b2a as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/6938f3b95bbc15aff8fb4a9d24d9aaf5/family1/cf56fda2084b4736b3389b67629c9b2a 2023-02-11 03:55:42,474 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/6938f3b95bbc15aff8fb4a9d24d9aaf5/family1/cf56fda2084b4736b3389b67629c9b2a, entries=4, sequenceid=5, filesize=5.4 K 2023-02-11 03:55:42,475 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/6938f3b95bbc15aff8fb4a9d24d9aaf5/.tmp/family2/9ec3c577685c478c85883a8dfde7d662 as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/6938f3b95bbc15aff8fb4a9d24d9aaf5/family2/9ec3c577685c478c85883a8dfde7d662 2023-02-11 03:55:42,478 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/6938f3b95bbc15aff8fb4a9d24d9aaf5/family2/9ec3c577685c478c85883a8dfde7d662, entries=2, sequenceid=5, filesize=5.1 K 2023-02-11 03:55:42,479 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for 6938f3b95bbc15aff8fb4a9d24d9aaf5 in 864ms, sequenceid=5, compaction requested=false 2023-02-11 03:55:42,479 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'ns:testMinorCompaction' 2023-02-11 03:55:42,479 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 6938f3b95bbc15aff8fb4a9d24d9aaf5: 2023-02-11 03:55:42,479 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,,1676087739490.6938f3b95bbc15aff8fb4a9d24d9aaf5. 2023-02-11 03:55:42,479 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-02-11 03:55:42,849 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230211f949b6dae7d04ee496e0a6c032a66ffb to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c81e728d9d4c2f636f067f89cc14862c20230211f949b6dae7d04ee496e0a6c032a66ffb 2023-02-11 03:55:42,865 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/3fa91eb317d63eaff7b1021985f3ad00/.tmp/family2/f3d9cf4338244975a22488a625861fd5 2023-02-11 03:55:42,868 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/3fa91eb317d63eaff7b1021985f3ad00/.tmp/family1/4da5a5dcbded4cb9ad85833159091832 as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/3fa91eb317d63eaff7b1021985f3ad00/family1/4da5a5dcbded4cb9ad85833159091832 2023-02-11 03:55:42,871 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/3fa91eb317d63eaff7b1021985f3ad00/family1/4da5a5dcbded4cb9ad85833159091832, entries=4, sequenceid=5, filesize=5.4 K 2023-02-11 03:55:42,872 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/3fa91eb317d63eaff7b1021985f3ad00/.tmp/family2/f3d9cf4338244975a22488a625861fd5 as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/3fa91eb317d63eaff7b1021985f3ad00/family2/f3d9cf4338244975a22488a625861fd5 2023-02-11 03:55:42,874 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/3fa91eb317d63eaff7b1021985f3ad00/family2/f3d9cf4338244975a22488a625861fd5, entries=2, sequenceid=5, filesize=5.1 K 2023-02-11 03:55:42,875 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-1] regionserver.HRegion(2947): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for 3fa91eb317d63eaff7b1021985f3ad00 in 1260ms, sequenceid=5, compaction requested=false 2023-02-11 03:55:42,876 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-1] regionserver.HRegion(2445): Flush status journal for 3fa91eb317d63eaff7b1021985f3ad00: 2023-02-11 03:55:42,876 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,1,1676087739490.3fa91eb317d63eaff7b1021985f3ad00. 2023-02-11 03:55:42,876 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-02-11 03:55:43,254 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/08cc9e8540da3c04f9eb90b5ad80782a/.tmp/family2/40cb6bf4cd204ced9ed8f8b62e7eb74b 2023-02-11 03:55:43,258 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/08cc9e8540da3c04f9eb90b5ad80782a/.tmp/family1/1965537a06594723af60377479103e74 as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/08cc9e8540da3c04f9eb90b5ad80782a/family1/1965537a06594723af60377479103e74 2023-02-11 03:55:43,262 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/08cc9e8540da3c04f9eb90b5ad80782a/family1/1965537a06594723af60377479103e74, entries=4, sequenceid=5, filesize=5.4 K 2023-02-11 03:55:43,263 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/08cc9e8540da3c04f9eb90b5ad80782a/.tmp/family2/40cb6bf4cd204ced9ed8f8b62e7eb74b as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/08cc9e8540da3c04f9eb90b5ad80782a/family2/40cb6bf4cd204ced9ed8f8b62e7eb74b 2023-02-11 03:55:43,266 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/08cc9e8540da3c04f9eb90b5ad80782a/family2/40cb6bf4cd204ced9ed8f8b62e7eb74b, entries=2, sequenceid=5, filesize=5.1 K 2023-02-11 03:55:43,267 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-2] regionserver.HRegion(2947): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for 08cc9e8540da3c04f9eb90b5ad80782a in 1652ms, sequenceid=5, compaction requested=false 2023-02-11 03:55:43,267 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-2] regionserver.HRegion(2445): Flush status journal for 08cc9e8540da3c04f9eb90b5ad80782a: 2023-02-11 03:55:43,267 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,2,1676087739490.08cc9e8540da3c04f9eb90b5ad80782a. 2023-02-11 03:55:43,267 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-02-11 03:55:43,267 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-02-11 03:55:43,267 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:55:43,267 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'ns:testMinorCompaction' locally acquired 2023-02-11 03:55:43,267 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37011,1676087618428' joining acquired barrier for procedure (ns:testMinorCompaction) in zk 2023-02-11 03:55:43,269 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:55:43,269 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-11 03:55:43,269 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:55:43,269 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:55:43,269 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:55:43,269 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-11 03:55:43,269 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'ns:testMinorCompaction' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 03:55:43,269 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:55:43,270 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:55:43,270 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-11 03:55:43,270 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:55:43,270 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:55:43,273 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37011,1676087618428' joining acquired barrier for procedure 'ns:testMinorCompaction' on coordinator 2023-02-11 03:55:43,273 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@759274e8[Count = 0] remaining members to acquire global barrier 2023-02-11 03:55:43,273 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'ns:testMinorCompaction' starting 'in-barrier' execution. 2023-02-11 03:55:43,273 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-11 03:55:43,274 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-11 03:55:43,274 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-11 03:55:43,274 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-11 03:55:43,274 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'ns:testMinorCompaction' received 'reached' from coordinator. 2023-02-11 03:55:43,274 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'ns:testMinorCompaction' locally completed 2023-02-11 03:55:43,274 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'ns:testMinorCompaction' completed for member 'jenkins-hbase4.apache.org,37011,1676087618428' in zk 2023-02-11 03:55:43,275 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:55:43,275 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 03:55:43,277 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:55:43,277 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'ns:testMinorCompaction' has notified controller of completion 2023-02-11 03:55:43,277 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:55:43,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:55:43,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:55:43,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:55:43,277 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'ns:testMinorCompaction' completed. 2023-02-11 03:55:43,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:55:43,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:55:43,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-11 03:55:43,279 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:55:43,279 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:55:43,279 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-11 03:55:43,279 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:55:43,280 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'ns:testMinorCompaction' member 'jenkins-hbase4.apache.org,37011,1676087618428': 2023-02-11 03:55:43,280 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37011,1676087618428' released barrier for procedure'ns:testMinorCompaction', counting down latch. Waiting for 0 more 2023-02-11 03:55:43,280 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'ns:testMinorCompaction' execution completed 2023-02-11 03:55:43,280 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 03:55:43,280 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 03:55:43,280 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:ns:testMinorCompaction 2023-02-11 03:55:43,280 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure ns:testMinorCompactionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 03:55:43,282 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-11 03:55:43,282 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-11 03:55:43,282 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-11 03:55:43,282 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:55:43,282 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-11 03:55:43,282 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-11 03:55:43,282 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:55:43,282 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:55:43,282 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:55:43,282 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:55:43,282 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:55:43,283 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:55:43,283 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-11 03:55:43,283 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-11 03:55:43,283 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:55:43,283 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-11 03:55:43,284 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:55:43,284 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:55:43,284 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:55:43,284 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-11 03:55:43,285 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:55:43,286 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:55:43,286 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:55:43,286 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:55:43,286 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-02-11 03:55:43,286 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'ns:testMinorCompaction' 2023-02-11 03:55:43,286 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 03:55:43,286 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:55:43,286 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:55:43,286 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-02-11 03:55:43,286 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:55:43,286 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:55:43,287 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-11 03:55:43,287 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : ns:testMinorCompaction'' to complete. (max 300000 ms per retry) 2023-02-11 03:55:43,287 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-11 03:55:43,287 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 03:55:43,287 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:55:43,287 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-11 03:55:43,287 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:55:43,387 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 03:55:43,388 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 03:55:43,389 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37011] regionserver.HRegion(8801): writing data to region ns:testMinorCompaction,,1676087739490.6938f3b95bbc15aff8fb4a9d24d9aaf5. with WAL disabled. Data may be lost in the event of a crash. 2023-02-11 03:55:43,389 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37011] regionserver.HRegion(8801): writing data to region ns:testMinorCompaction,1,1676087739490.3fa91eb317d63eaff7b1021985f3ad00. with WAL disabled. Data may be lost in the event of a crash. 2023-02-11 03:55:43,389 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37011] regionserver.HRegion(8801): writing data to region ns:testMinorCompaction,2,1676087739490.08cc9e8540da3c04f9eb90b5ad80782a. with WAL disabled. Data may be lost in the event of a crash. 2023-02-11 03:55:43,391 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 03:55:43,393 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] procedure.ProcedureCoordinator(143): Procedure ns:testMinorCompaction was in running list but was completed. Accepting new attempt. 2023-02-11 03:55:43,393 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] procedure.ProcedureCoordinator(165): Submitting procedure ns:testMinorCompaction 2023-02-11 03:55:43,393 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'ns:testMinorCompaction' 2023-02-11 03:55:43,393 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:55:43,393 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'ns:testMinorCompaction' starting 'acquire' 2023-02-11 03:55:43,393 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'ns:testMinorCompaction', kicking off acquire phase on members. 2023-02-11 03:55:43,393 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-11 03:55:43,393 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-02-11 03:55:43,394 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:55:43,394 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:55:43,395 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:55:43,395 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:55:43,395 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:55:43,395 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 03:55:43,395 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-02-11 03:55:43,395 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-11 03:55:43,395 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 03:55:43,395 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-02-11 03:55:43,395 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for ns:testMinorCompaction 2023-02-11 03:55:43,396 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc ns:testMinorCompaction is still present, removing 2023-02-11 03:55:43,396 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:ns:testMinorCompaction 2023-02-11 03:55:43,396 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'ns:testMinorCompaction' with timeout 60000ms 2023-02-11 03:55:43,396 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:55:43,396 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'ns:testMinorCompaction' starting 'acquire' stage 2023-02-11 03:55:43,396 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,,1676087739490.6938f3b95bbc15aff8fb4a9d24d9aaf5. 2023-02-11 03:55:43,396 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,1,1676087739490.3fa91eb317d63eaff7b1021985f3ad00. 2023-02-11 03:55:43,396 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,2,1676087739490.08cc9e8540da3c04f9eb90b5ad80782a. 2023-02-11 03:55:43,396 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,2,1676087739490.08cc9e8540da3c04f9eb90b5ad80782a. started... 2023-02-11 03:55:43,396 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-02-11 03:55:43,396 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,1,1676087739490.3fa91eb317d63eaff7b1021985f3ad00. started... 2023-02-11 03:55:43,396 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,,1676087739490.6938f3b95bbc15aff8fb4a9d24d9aaf5. started... 2023-02-11 03:55:43,396 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-2] regionserver.HRegion(2744): Flushing 08cc9e8540da3c04f9eb90b5ad80782a 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-02-11 03:55:43,396 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-1] regionserver.HRegion(2744): Flushing 3fa91eb317d63eaff7b1021985f3ad00 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-02-11 03:55:43,396 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 6938f3b95bbc15aff8fb4a9d24d9aaf5 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-02-11 03:55:43,396 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 03:55:43,815 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202302118ff9a9892b31401a838f70329e7c1a43 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c81e728d9d4c2f636f067f89cc14862c202302118ff9a9892b31401a838f70329e7c1a43 2023-02-11 03:55:43,815 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230211c68ad96fc9b24d79ade519d4de142fec to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/d41d8cd98f00b204e9800998ecf8427e20230211c68ad96fc9b24d79ade519d4de142fec 2023-02-11 03:55:43,815 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202302113e659db2405c468fbea4997f9d4f11b7 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c4ca4238a0b923820dcc509a6f75849b202302113e659db2405c468fbea4997f9d4f11b7 2023-02-11 03:55:44,221 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/6938f3b95bbc15aff8fb4a9d24d9aaf5/.tmp/family1/3f86c29984af496d8a85fdfd4185e520 2023-02-11 03:55:44,221 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/3fa91eb317d63eaff7b1021985f3ad00/.tmp/family1/971239e058184febb54e467f537ea248 2023-02-11 03:55:44,221 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/08cc9e8540da3c04f9eb90b5ad80782a/.tmp/family1/42cb79fafc66413294674382953efdf2 2023-02-11 03:55:44,264 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230211871ab972f4f6470a9b47e3aea7756e5f to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/d41d8cd98f00b204e9800998ecf8427e20230211871ab972f4f6470a9b47e3aea7756e5f 2023-02-11 03:55:44,265 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202302116786bd45f97d44b8b92c4b7529b2e89e to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c4ca4238a0b923820dcc509a6f75849b202302116786bd45f97d44b8b92c4b7529b2e89e 2023-02-11 03:55:44,281 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/6938f3b95bbc15aff8fb4a9d24d9aaf5/.tmp/family2/9c5820d8e743439fab3b9790d24f84ce 2023-02-11 03:55:44,285 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230211a345cf40e6124683a86fb94f488aa88c to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c81e728d9d4c2f636f067f89cc14862c20230211a345cf40e6124683a86fb94f488aa88c 2023-02-11 03:55:44,286 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/6938f3b95bbc15aff8fb4a9d24d9aaf5/.tmp/family1/3f86c29984af496d8a85fdfd4185e520 as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/6938f3b95bbc15aff8fb4a9d24d9aaf5/family1/3f86c29984af496d8a85fdfd4185e520 2023-02-11 03:55:44,291 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/6938f3b95bbc15aff8fb4a9d24d9aaf5/family1/3f86c29984af496d8a85fdfd4185e520, entries=4, sequenceid=9, filesize=5.4 K 2023-02-11 03:55:44,292 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/6938f3b95bbc15aff8fb4a9d24d9aaf5/.tmp/family2/9c5820d8e743439fab3b9790d24f84ce as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/6938f3b95bbc15aff8fb4a9d24d9aaf5/family2/9c5820d8e743439fab3b9790d24f84ce 2023-02-11 03:55:44,293 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/08cc9e8540da3c04f9eb90b5ad80782a/.tmp/family2/2ae65cf0e4db479c8f62d84e1541cfe0 2023-02-11 03:55:44,297 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/6938f3b95bbc15aff8fb4a9d24d9aaf5/family2/9c5820d8e743439fab3b9790d24f84ce, entries=2, sequenceid=9, filesize=5.1 K 2023-02-11 03:55:44,298 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for 6938f3b95bbc15aff8fb4a9d24d9aaf5 in 902ms, sequenceid=9, compaction requested=false 2023-02-11 03:55:44,298 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/08cc9e8540da3c04f9eb90b5ad80782a/.tmp/family1/42cb79fafc66413294674382953efdf2 as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/08cc9e8540da3c04f9eb90b5ad80782a/family1/42cb79fafc66413294674382953efdf2 2023-02-11 03:55:44,298 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 6938f3b95bbc15aff8fb4a9d24d9aaf5: 2023-02-11 03:55:44,299 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,,1676087739490.6938f3b95bbc15aff8fb4a9d24d9aaf5. 2023-02-11 03:55:44,299 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-02-11 03:55:44,304 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/08cc9e8540da3c04f9eb90b5ad80782a/family1/42cb79fafc66413294674382953efdf2, entries=4, sequenceid=9, filesize=5.4 K 2023-02-11 03:55:44,305 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/08cc9e8540da3c04f9eb90b5ad80782a/.tmp/family2/2ae65cf0e4db479c8f62d84e1541cfe0 as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/08cc9e8540da3c04f9eb90b5ad80782a/family2/2ae65cf0e4db479c8f62d84e1541cfe0 2023-02-11 03:55:44,309 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/08cc9e8540da3c04f9eb90b5ad80782a/family2/2ae65cf0e4db479c8f62d84e1541cfe0, entries=2, sequenceid=9, filesize=5.1 K 2023-02-11 03:55:44,311 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-2] regionserver.HRegion(2947): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for 08cc9e8540da3c04f9eb90b5ad80782a in 915ms, sequenceid=9, compaction requested=false 2023-02-11 03:55:44,311 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-2] regionserver.HRegion(2445): Flush status journal for 08cc9e8540da3c04f9eb90b5ad80782a: 2023-02-11 03:55:44,311 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,2,1676087739490.08cc9e8540da3c04f9eb90b5ad80782a. 2023-02-11 03:55:44,311 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-02-11 03:55:44,684 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/3fa91eb317d63eaff7b1021985f3ad00/.tmp/family2/7750003a52504750aa8012b5abcdd4ee 2023-02-11 03:55:44,690 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/3fa91eb317d63eaff7b1021985f3ad00/.tmp/family1/971239e058184febb54e467f537ea248 as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/3fa91eb317d63eaff7b1021985f3ad00/family1/971239e058184febb54e467f537ea248 2023-02-11 03:55:44,693 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/3fa91eb317d63eaff7b1021985f3ad00/family1/971239e058184febb54e467f537ea248, entries=4, sequenceid=9, filesize=5.4 K 2023-02-11 03:55:44,694 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/3fa91eb317d63eaff7b1021985f3ad00/.tmp/family2/7750003a52504750aa8012b5abcdd4ee as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/3fa91eb317d63eaff7b1021985f3ad00/family2/7750003a52504750aa8012b5abcdd4ee 2023-02-11 03:55:44,698 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/3fa91eb317d63eaff7b1021985f3ad00/family2/7750003a52504750aa8012b5abcdd4ee, entries=2, sequenceid=9, filesize=5.1 K 2023-02-11 03:55:44,698 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-1] regionserver.HRegion(2947): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for 3fa91eb317d63eaff7b1021985f3ad00 in 1302ms, sequenceid=9, compaction requested=false 2023-02-11 03:55:44,698 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-1] regionserver.HRegion(2445): Flush status journal for 3fa91eb317d63eaff7b1021985f3ad00: 2023-02-11 03:55:44,698 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,1,1676087739490.3fa91eb317d63eaff7b1021985f3ad00. 2023-02-11 03:55:44,699 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-02-11 03:55:44,699 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-02-11 03:55:44,699 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:55:44,699 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'ns:testMinorCompaction' locally acquired 2023-02-11 03:55:44,699 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37011,1676087618428' joining acquired barrier for procedure (ns:testMinorCompaction) in zk 2023-02-11 03:55:44,701 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-11 03:55:44,701 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:55:44,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:55:44,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:55:44,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:55:44,701 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-11 03:55:44,701 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'ns:testMinorCompaction' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 03:55:44,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:55:44,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:55:44,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-11 03:55:44,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:55:44,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:55:44,703 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37011,1676087618428' joining acquired barrier for procedure 'ns:testMinorCompaction' on coordinator 2023-02-11 03:55:44,703 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1e4135f4[Count = 0] remaining members to acquire global barrier 2023-02-11 03:55:44,703 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'ns:testMinorCompaction' starting 'in-barrier' execution. 2023-02-11 03:55:44,703 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-11 03:55:44,704 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-11 03:55:44,704 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-11 03:55:44,704 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-11 03:55:44,704 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'ns:testMinorCompaction' received 'reached' from coordinator. 2023-02-11 03:55:44,704 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'ns:testMinorCompaction' locally completed 2023-02-11 03:55:44,704 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'ns:testMinorCompaction' completed for member 'jenkins-hbase4.apache.org,37011,1676087618428' in zk 2023-02-11 03:55:44,704 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:55:44,704 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 03:55:44,708 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:55:44,708 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'ns:testMinorCompaction' has notified controller of completion 2023-02-11 03:55:44,708 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:55:44,708 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:55:44,708 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:55:44,708 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:55:44,708 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'ns:testMinorCompaction' completed. 2023-02-11 03:55:44,709 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:55:44,709 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:55:44,709 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-11 03:55:44,710 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:55:44,710 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:55:44,711 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-11 03:55:44,711 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:55:44,712 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'ns:testMinorCompaction' member 'jenkins-hbase4.apache.org,37011,1676087618428': 2023-02-11 03:55:44,712 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37011,1676087618428' released barrier for procedure'ns:testMinorCompaction', counting down latch. Waiting for 0 more 2023-02-11 03:55:44,712 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'ns:testMinorCompaction' execution completed 2023-02-11 03:55:44,712 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 03:55:44,712 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 03:55:44,712 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:ns:testMinorCompaction 2023-02-11 03:55:44,712 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure ns:testMinorCompactionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 03:55:44,714 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-11 03:55:44,714 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-11 03:55:44,714 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:55:44,714 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:55:44,714 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-11 03:55:44,714 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:55:44,714 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-11 03:55:44,714 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-11 03:55:44,714 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:55:44,714 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:55:44,714 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:55:44,714 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:55:44,715 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-11 03:55:44,715 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-11 03:55:44,715 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:55:44,716 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:55:44,720 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-11 03:55:44,720 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:55:44,721 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:55:44,721 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-11 03:55:44,722 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:55:44,722 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:55:44,722 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:55:44,722 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'ns:testMinorCompaction' 2023-02-11 03:55:44,722 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 03:55:44,722 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:55:44,722 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/ns:testMinorCompaction because node does not exist (not an error) 2023-02-11 03:55:44,722 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:55:44,722 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:55:44,722 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : ns:testMinorCompaction'' to complete. (max 300000 ms per retry) 2023-02-11 03:55:44,722 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-02-11 03:55:44,722 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 03:55:44,723 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:55:44,722 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-02-11 03:55:44,723 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:55:44,723 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:55:44,723 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-11 03:55:44,723 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-11 03:55:44,723 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-11 03:55:44,823 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 03:55:44,823 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 03:55:44,824 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37011] regionserver.HRegion(8801): writing data to region ns:testMinorCompaction,,1676087739490.6938f3b95bbc15aff8fb4a9d24d9aaf5. with WAL disabled. Data may be lost in the event of a crash. 2023-02-11 03:55:44,825 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37011] regionserver.HRegion(8801): writing data to region ns:testMinorCompaction,1,1676087739490.3fa91eb317d63eaff7b1021985f3ad00. with WAL disabled. Data may be lost in the event of a crash. 2023-02-11 03:55:44,825 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37011] regionserver.HRegion(8801): writing data to region ns:testMinorCompaction,2,1676087739490.08cc9e8540da3c04f9eb90b5ad80782a. with WAL disabled. Data may be lost in the event of a crash. 2023-02-11 03:55:44,827 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 03:55:44,828 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] procedure.ProcedureCoordinator(143): Procedure ns:testMinorCompaction was in running list but was completed. Accepting new attempt. 2023-02-11 03:55:44,828 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] procedure.ProcedureCoordinator(165): Submitting procedure ns:testMinorCompaction 2023-02-11 03:55:44,828 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'ns:testMinorCompaction' 2023-02-11 03:55:44,828 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:55:44,829 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'ns:testMinorCompaction' starting 'acquire' 2023-02-11 03:55:44,829 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'ns:testMinorCompaction', kicking off acquire phase on members. 2023-02-11 03:55:44,829 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-11 03:55:44,829 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-02-11 03:55:44,832 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:55:44,832 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:55:44,832 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:55:44,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:55:44,832 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:55:44,832 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 03:55:44,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-02-11 03:55:44,832 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-11 03:55:44,833 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 03:55:44,833 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-02-11 03:55:44,833 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for ns:testMinorCompaction 2023-02-11 03:55:44,833 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc ns:testMinorCompaction is still present, removing 2023-02-11 03:55:44,833 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:ns:testMinorCompaction 2023-02-11 03:55:44,833 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'ns:testMinorCompaction' with timeout 60000ms 2023-02-11 03:55:44,833 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:55:44,833 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'ns:testMinorCompaction' starting 'acquire' stage 2023-02-11 03:55:44,833 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,,1676087739490.6938f3b95bbc15aff8fb4a9d24d9aaf5. 2023-02-11 03:55:44,833 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,,1676087739490.6938f3b95bbc15aff8fb4a9d24d9aaf5. started... 2023-02-11 03:55:44,833 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,1,1676087739490.3fa91eb317d63eaff7b1021985f3ad00. 2023-02-11 03:55:44,833 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,1,1676087739490.3fa91eb317d63eaff7b1021985f3ad00. started... 2023-02-11 03:55:44,833 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-02-11 03:55:44,833 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 6938f3b95bbc15aff8fb4a9d24d9aaf5 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-02-11 03:55:44,833 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-1] regionserver.HRegion(2744): Flushing 3fa91eb317d63eaff7b1021985f3ad00 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-02-11 03:55:44,833 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 03:55:44,833 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,2,1676087739490.08cc9e8540da3c04f9eb90b5ad80782a. 2023-02-11 03:55:44,834 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,2,1676087739490.08cc9e8540da3c04f9eb90b5ad80782a. started... 2023-02-11 03:55:44,834 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-2] regionserver.HRegion(2744): Flushing 08cc9e8540da3c04f9eb90b5ad80782a 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-02-11 03:55:44,849 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202302118b66ca51c7c344dd85d6f5d89ec85a10 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c4ca4238a0b923820dcc509a6f75849b202302118b66ca51c7c344dd85d6f5d89ec85a10 2023-02-11 03:55:45,249 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202302110646c375070f47988c0809e864310997 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/d41d8cd98f00b204e9800998ecf8427e202302110646c375070f47988c0809e864310997 2023-02-11 03:55:45,250 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230211910fac6d05994e5f8025934e4415e093 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c81e728d9d4c2f636f067f89cc14862c20230211910fac6d05994e5f8025934e4415e093 2023-02-11 03:55:45,259 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/3fa91eb317d63eaff7b1021985f3ad00/.tmp/family1/2ad4cd5d0be44732a4a22dd4489bde40 2023-02-11 03:55:45,655 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/6938f3b95bbc15aff8fb4a9d24d9aaf5/.tmp/family1/74afb6ba1d1348a09ccd3bbccc4f4536 2023-02-11 03:55:45,661 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/08cc9e8540da3c04f9eb90b5ad80782a/.tmp/family1/c44faad7b3744429af39663af5118d17 2023-02-11 03:55:45,671 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230211ccdb4b64bbfd40b4aac72f9637a7cb9e to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c4ca4238a0b923820dcc509a6f75849b20230211ccdb4b64bbfd40b4aac72f9637a7cb9e 2023-02-11 03:55:45,672 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202302114f7c6f1bb6ad42d89e2a73dae32f70a6 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c81e728d9d4c2f636f067f89cc14862c202302114f7c6f1bb6ad42d89e2a73dae32f70a6 2023-02-11 03:55:46,067 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2023021184fa830ed8fd43208aee389d9a7d82af to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/d41d8cd98f00b204e9800998ecf8427e2023021184fa830ed8fd43208aee389d9a7d82af 2023-02-11 03:55:46,076 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/3fa91eb317d63eaff7b1021985f3ad00/.tmp/family2/5bc433a5b8d7447bb75bfea1ccc3c50c 2023-02-11 03:55:46,076 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/08cc9e8540da3c04f9eb90b5ad80782a/.tmp/family2/be792f70fa374c46a3ecacbbae329638 2023-02-11 03:55:46,079 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/3fa91eb317d63eaff7b1021985f3ad00/.tmp/family1/2ad4cd5d0be44732a4a22dd4489bde40 as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/3fa91eb317d63eaff7b1021985f3ad00/family1/2ad4cd5d0be44732a4a22dd4489bde40 2023-02-11 03:55:46,079 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/08cc9e8540da3c04f9eb90b5ad80782a/.tmp/family1/c44faad7b3744429af39663af5118d17 as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/08cc9e8540da3c04f9eb90b5ad80782a/family1/c44faad7b3744429af39663af5118d17 2023-02-11 03:55:46,082 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/3fa91eb317d63eaff7b1021985f3ad00/family1/2ad4cd5d0be44732a4a22dd4489bde40, entries=4, sequenceid=13, filesize=5.4 K 2023-02-11 03:55:46,083 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/08cc9e8540da3c04f9eb90b5ad80782a/family1/c44faad7b3744429af39663af5118d17, entries=4, sequenceid=13, filesize=5.4 K 2023-02-11 03:55:46,083 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/3fa91eb317d63eaff7b1021985f3ad00/.tmp/family2/5bc433a5b8d7447bb75bfea1ccc3c50c as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/3fa91eb317d63eaff7b1021985f3ad00/family2/5bc433a5b8d7447bb75bfea1ccc3c50c 2023-02-11 03:55:46,083 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/08cc9e8540da3c04f9eb90b5ad80782a/.tmp/family2/be792f70fa374c46a3ecacbbae329638 as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/08cc9e8540da3c04f9eb90b5ad80782a/family2/be792f70fa374c46a3ecacbbae329638 2023-02-11 03:55:46,087 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/3fa91eb317d63eaff7b1021985f3ad00/family2/5bc433a5b8d7447bb75bfea1ccc3c50c, entries=2, sequenceid=13, filesize=5.1 K 2023-02-11 03:55:46,087 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/08cc9e8540da3c04f9eb90b5ad80782a/family2/be792f70fa374c46a3ecacbbae329638, entries=2, sequenceid=13, filesize=5.1 K 2023-02-11 03:55:46,087 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-1] regionserver.HRegion(2947): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for 3fa91eb317d63eaff7b1021985f3ad00 in 1254ms, sequenceid=13, compaction requested=true 2023-02-11 03:55:46,087 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-2] regionserver.HRegion(2947): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for 08cc9e8540da3c04f9eb90b5ad80782a in 1253ms, sequenceid=13, compaction requested=true 2023-02-11 03:55:46,087 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-1] regionserver.HRegion(2445): Flush status journal for 3fa91eb317d63eaff7b1021985f3ad00: 2023-02-11 03:55:46,087 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-2] regionserver.HRegion(2445): Flush status journal for 08cc9e8540da3c04f9eb90b5ad80782a: 2023-02-11 03:55:46,087 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,1,1676087739490.3fa91eb317d63eaff7b1021985f3ad00. 2023-02-11 03:55:46,087 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,2,1676087739490.08cc9e8540da3c04f9eb90b5ad80782a. 2023-02-11 03:55:46,087 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-02-11 03:55:46,088 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-02-11 03:55:46,138 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-11 03:55:46,472 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/6938f3b95bbc15aff8fb4a9d24d9aaf5/.tmp/family2/d981d090bce74425ba2cbbcdded59840 2023-02-11 03:55:46,476 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/6938f3b95bbc15aff8fb4a9d24d9aaf5/.tmp/family1/74afb6ba1d1348a09ccd3bbccc4f4536 as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/6938f3b95bbc15aff8fb4a9d24d9aaf5/family1/74afb6ba1d1348a09ccd3bbccc4f4536 2023-02-11 03:55:46,480 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/6938f3b95bbc15aff8fb4a9d24d9aaf5/family1/74afb6ba1d1348a09ccd3bbccc4f4536, entries=4, sequenceid=13, filesize=5.4 K 2023-02-11 03:55:46,480 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/6938f3b95bbc15aff8fb4a9d24d9aaf5/.tmp/family2/d981d090bce74425ba2cbbcdded59840 as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/6938f3b95bbc15aff8fb4a9d24d9aaf5/family2/d981d090bce74425ba2cbbcdded59840 2023-02-11 03:55:46,483 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/6938f3b95bbc15aff8fb4a9d24d9aaf5/family2/d981d090bce74425ba2cbbcdded59840, entries=2, sequenceid=13, filesize=5.1 K 2023-02-11 03:55:46,484 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for 6938f3b95bbc15aff8fb4a9d24d9aaf5 in 1651ms, sequenceid=13, compaction requested=true 2023-02-11 03:55:46,484 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 6938f3b95bbc15aff8fb4a9d24d9aaf5: 2023-02-11 03:55:46,484 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,,1676087739490.6938f3b95bbc15aff8fb4a9d24d9aaf5. 2023-02-11 03:55:46,484 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-02-11 03:55:46,485 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-02-11 03:55:46,485 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:55:46,485 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'ns:testMinorCompaction' locally acquired 2023-02-11 03:55:46,485 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37011,1676087618428' joining acquired barrier for procedure (ns:testMinorCompaction) in zk 2023-02-11 03:55:46,487 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-11 03:55:46,487 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:55:46,488 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:55:46,488 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:55:46,488 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:55:46,488 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-11 03:55:46,488 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'ns:testMinorCompaction' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 03:55:46,488 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:55:46,488 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:55:46,488 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-11 03:55:46,489 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:55:46,489 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:55:46,489 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37011,1676087618428' joining acquired barrier for procedure 'ns:testMinorCompaction' on coordinator 2023-02-11 03:55:46,489 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7ccc5ce5[Count = 0] remaining members to acquire global barrier 2023-02-11 03:55:46,489 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'ns:testMinorCompaction' starting 'in-barrier' execution. 2023-02-11 03:55:46,489 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-11 03:55:46,491 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-11 03:55:46,491 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-11 03:55:46,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-11 03:55:46,491 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'ns:testMinorCompaction' received 'reached' from coordinator. 2023-02-11 03:55:46,491 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'ns:testMinorCompaction' locally completed 2023-02-11 03:55:46,491 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'ns:testMinorCompaction' completed for member 'jenkins-hbase4.apache.org,37011,1676087618428' in zk 2023-02-11 03:55:46,491 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:55:46,491 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 03:55:46,492 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'ns:testMinorCompaction' has notified controller of completion 2023-02-11 03:55:46,492 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:55:46,492 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:55:46,492 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:55:46,492 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:55:46,492 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:55:46,492 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'ns:testMinorCompaction' completed. 2023-02-11 03:55:46,492 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:55:46,493 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:55:46,493 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-11 03:55:46,493 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:55:46,493 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:55:46,494 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-11 03:55:46,494 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:55:46,494 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'ns:testMinorCompaction' member 'jenkins-hbase4.apache.org,37011,1676087618428': 2023-02-11 03:55:46,494 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37011,1676087618428' released barrier for procedure'ns:testMinorCompaction', counting down latch. Waiting for 0 more 2023-02-11 03:55:46,494 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'ns:testMinorCompaction' execution completed 2023-02-11 03:55:46,494 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 03:55:46,494 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 03:55:46,494 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:ns:testMinorCompaction 2023-02-11 03:55:46,494 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure ns:testMinorCompactionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 03:55:46,496 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-11 03:55:46,496 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-11 03:55:46,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-11 03:55:46,496 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-11 03:55:46,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:55:46,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:55:46,496 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:55:46,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-11 03:55:46,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:55:46,496 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:55:46,496 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:55:46,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:55:46,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-11 03:55:46,497 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-11 03:55:46,497 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:55:46,497 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-11 03:55:46,497 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:55:46,497 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:55:46,497 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:55:46,498 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-11 03:55:46,498 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:55:46,504 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:55:46,504 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:55:46,504 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-02-11 03:55:46,504 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:55:46,505 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:55:46,504 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:55:46,504 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'ns:testMinorCompaction' 2023-02-11 03:55:46,505 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 03:55:46,504 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-02-11 03:55:46,504 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:55:46,505 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:55:46,505 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:55:46,505 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:55:46,505 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-11 03:55:46,505 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-11 03:55:46,505 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : ns:testMinorCompaction'' to complete. (max 300000 ms per retry) 2023-02-11 03:55:46,505 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 03:55:46,505 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-11 03:55:46,605 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 03:55:46,606 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 03:55:46,607 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37011] regionserver.HRegion(8801): writing data to region ns:testMinorCompaction,,1676087739490.6938f3b95bbc15aff8fb4a9d24d9aaf5. with WAL disabled. Data may be lost in the event of a crash. 2023-02-11 03:55:46,607 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37011] regionserver.HRegion(8801): writing data to region ns:testMinorCompaction,1,1676087739490.3fa91eb317d63eaff7b1021985f3ad00. with WAL disabled. Data may be lost in the event of a crash. 2023-02-11 03:55:46,607 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37011] regionserver.HRegion(8801): writing data to region ns:testMinorCompaction,2,1676087739490.08cc9e8540da3c04f9eb90b5ad80782a. with WAL disabled. Data may be lost in the event of a crash. 2023-02-11 03:55:46,609 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 03:55:46,611 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] procedure.ProcedureCoordinator(143): Procedure ns:testMinorCompaction was in running list but was completed. Accepting new attempt. 2023-02-11 03:55:46,611 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] procedure.ProcedureCoordinator(165): Submitting procedure ns:testMinorCompaction 2023-02-11 03:55:46,611 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'ns:testMinorCompaction' 2023-02-11 03:55:46,611 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:55:46,611 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'ns:testMinorCompaction' starting 'acquire' 2023-02-11 03:55:46,611 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'ns:testMinorCompaction', kicking off acquire phase on members. 2023-02-11 03:55:46,611 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-11 03:55:46,611 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-02-11 03:55:46,613 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:55:46,613 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:55:46,613 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:55:46,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:55:46,613 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:55:46,613 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 03:55:46,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-02-11 03:55:46,613 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-11 03:55:46,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 03:55:46,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-02-11 03:55:46,614 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for ns:testMinorCompaction 2023-02-11 03:55:46,614 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc ns:testMinorCompaction is still present, removing 2023-02-11 03:55:46,614 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:ns:testMinorCompaction 2023-02-11 03:55:46,614 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'ns:testMinorCompaction' with timeout 60000ms 2023-02-11 03:55:46,614 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:55:46,614 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'ns:testMinorCompaction' starting 'acquire' stage 2023-02-11 03:55:46,614 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,,1676087739490.6938f3b95bbc15aff8fb4a9d24d9aaf5. 2023-02-11 03:55:46,614 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,,1676087739490.6938f3b95bbc15aff8fb4a9d24d9aaf5. started... 2023-02-11 03:55:46,614 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,1,1676087739490.3fa91eb317d63eaff7b1021985f3ad00. 2023-02-11 03:55:46,614 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,2,1676087739490.08cc9e8540da3c04f9eb90b5ad80782a. 2023-02-11 03:55:46,614 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,2,1676087739490.08cc9e8540da3c04f9eb90b5ad80782a. started... 2023-02-11 03:55:46,614 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 6938f3b95bbc15aff8fb4a9d24d9aaf5 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-02-11 03:55:46,614 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-02-11 03:55:46,614 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 03:55:46,614 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-2] regionserver.HRegion(2744): Flushing 08cc9e8540da3c04f9eb90b5ad80782a 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-02-11 03:55:46,614 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,1,1676087739490.3fa91eb317d63eaff7b1021985f3ad00. started... 2023-02-11 03:55:46,615 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-1] regionserver.HRegion(2744): Flushing 3fa91eb317d63eaff7b1021985f3ad00 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-02-11 03:55:47,026 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c2023021172c45c87709247b3adb64fe112fb7a0b to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c81e728d9d4c2f636f067f89cc14862c2023021172c45c87709247b3adb64fe112fb7a0b 2023-02-11 03:55:47,026 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230211b5d1211b2dc64824adb36e500248b866 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c4ca4238a0b923820dcc509a6f75849b20230211b5d1211b2dc64824adb36e500248b866 2023-02-11 03:55:47,026 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230211991b2a62287842d6b4270d66e520b307 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/d41d8cd98f00b204e9800998ecf8427e20230211991b2a62287842d6b4270d66e520b307 2023-02-11 03:55:47,434 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/6938f3b95bbc15aff8fb4a9d24d9aaf5/.tmp/family1/452e95bc02544b499b6dcea6dae339ec 2023-02-11 03:55:47,434 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/3fa91eb317d63eaff7b1021985f3ad00/.tmp/family1/4ae66d741c9441a180df4ce64e5983a1 2023-02-11 03:55:47,434 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/08cc9e8540da3c04f9eb90b5ad80782a/.tmp/family1/e607cd8b09b34b92bd7f72d6940e698d 2023-02-11 03:55:47,453 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230211335cc8335fa446f49b62a36b6c563a47 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c4ca4238a0b923820dcc509a6f75849b20230211335cc8335fa446f49b62a36b6c563a47 2023-02-11 03:55:47,853 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c2023021187b209fca0644af4bb2341f942d74c84 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c81e728d9d4c2f636f067f89cc14862c2023021187b209fca0644af4bb2341f942d74c84 2023-02-11 03:55:47,854 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230211c5c1ff7c79d44fbebd32f42ac604dc1a to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/d41d8cd98f00b204e9800998ecf8427e20230211c5c1ff7c79d44fbebd32f42ac604dc1a 2023-02-11 03:55:47,857 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/3fa91eb317d63eaff7b1021985f3ad00/.tmp/family2/c4be4f3faf124a13ab03497afaac436c 2023-02-11 03:55:47,861 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/3fa91eb317d63eaff7b1021985f3ad00/.tmp/family1/4ae66d741c9441a180df4ce64e5983a1 as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/3fa91eb317d63eaff7b1021985f3ad00/family1/4ae66d741c9441a180df4ce64e5983a1 2023-02-11 03:55:47,864 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/3fa91eb317d63eaff7b1021985f3ad00/family1/4ae66d741c9441a180df4ce64e5983a1, entries=4, sequenceid=17, filesize=5.4 K 2023-02-11 03:55:47,864 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/3fa91eb317d63eaff7b1021985f3ad00/.tmp/family2/c4be4f3faf124a13ab03497afaac436c as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/3fa91eb317d63eaff7b1021985f3ad00/family2/c4be4f3faf124a13ab03497afaac436c 2023-02-11 03:55:47,867 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/3fa91eb317d63eaff7b1021985f3ad00/family2/c4be4f3faf124a13ab03497afaac436c, entries=2, sequenceid=17, filesize=5.1 K 2023-02-11 03:55:47,868 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-1] regionserver.HRegion(2947): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for 3fa91eb317d63eaff7b1021985f3ad00 in 1253ms, sequenceid=17, compaction requested=true 2023-02-11 03:55:47,868 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-1] regionserver.HRegion(2445): Flush status journal for 3fa91eb317d63eaff7b1021985f3ad00: 2023-02-11 03:55:47,868 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,1,1676087739490.3fa91eb317d63eaff7b1021985f3ad00. 2023-02-11 03:55:47,868 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-02-11 03:55:48,259 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/08cc9e8540da3c04f9eb90b5ad80782a/.tmp/family2/fa32da4a61cf4a7794826f9b7f890709 2023-02-11 03:55:48,259 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/6938f3b95bbc15aff8fb4a9d24d9aaf5/.tmp/family2/208d33478a60482482546463e56b36ce 2023-02-11 03:55:48,263 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/08cc9e8540da3c04f9eb90b5ad80782a/.tmp/family1/e607cd8b09b34b92bd7f72d6940e698d as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/08cc9e8540da3c04f9eb90b5ad80782a/family1/e607cd8b09b34b92bd7f72d6940e698d 2023-02-11 03:55:48,263 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/6938f3b95bbc15aff8fb4a9d24d9aaf5/.tmp/family1/452e95bc02544b499b6dcea6dae339ec as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/6938f3b95bbc15aff8fb4a9d24d9aaf5/family1/452e95bc02544b499b6dcea6dae339ec 2023-02-11 03:55:48,266 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/08cc9e8540da3c04f9eb90b5ad80782a/family1/e607cd8b09b34b92bd7f72d6940e698d, entries=4, sequenceid=17, filesize=5.4 K 2023-02-11 03:55:48,266 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/6938f3b95bbc15aff8fb4a9d24d9aaf5/family1/452e95bc02544b499b6dcea6dae339ec, entries=4, sequenceid=17, filesize=5.4 K 2023-02-11 03:55:48,266 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/08cc9e8540da3c04f9eb90b5ad80782a/.tmp/family2/fa32da4a61cf4a7794826f9b7f890709 as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/08cc9e8540da3c04f9eb90b5ad80782a/family2/fa32da4a61cf4a7794826f9b7f890709 2023-02-11 03:55:48,267 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/6938f3b95bbc15aff8fb4a9d24d9aaf5/.tmp/family2/208d33478a60482482546463e56b36ce as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/6938f3b95bbc15aff8fb4a9d24d9aaf5/family2/208d33478a60482482546463e56b36ce 2023-02-11 03:55:48,270 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/08cc9e8540da3c04f9eb90b5ad80782a/family2/fa32da4a61cf4a7794826f9b7f890709, entries=2, sequenceid=17, filesize=5.1 K 2023-02-11 03:55:48,270 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/6938f3b95bbc15aff8fb4a9d24d9aaf5/family2/208d33478a60482482546463e56b36ce, entries=2, sequenceid=17, filesize=5.1 K 2023-02-11 03:55:48,271 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-2] regionserver.HRegion(2947): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for 08cc9e8540da3c04f9eb90b5ad80782a in 1656ms, sequenceid=17, compaction requested=true 2023-02-11 03:55:48,271 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-2] regionserver.HRegion(2445): Flush status journal for 08cc9e8540da3c04f9eb90b5ad80782a: 2023-02-11 03:55:48,271 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,2,1676087739490.08cc9e8540da3c04f9eb90b5ad80782a. 2023-02-11 03:55:48,271 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-02-11 03:55:48,271 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for 6938f3b95bbc15aff8fb4a9d24d9aaf5 in 1657ms, sequenceid=17, compaction requested=true 2023-02-11 03:55:48,271 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 6938f3b95bbc15aff8fb4a9d24d9aaf5: 2023-02-11 03:55:48,271 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,,1676087739490.6938f3b95bbc15aff8fb4a9d24d9aaf5. 2023-02-11 03:55:48,271 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-02-11 03:55:48,271 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-02-11 03:55:48,271 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:55:48,271 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'ns:testMinorCompaction' locally acquired 2023-02-11 03:55:48,271 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37011,1676087618428' joining acquired barrier for procedure (ns:testMinorCompaction) in zk 2023-02-11 03:55:48,273 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:55:48,273 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-11 03:55:48,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:55:48,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:55:48,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:55:48,273 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-11 03:55:48,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:55:48,273 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'ns:testMinorCompaction' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 03:55:48,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:55:48,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-11 03:55:48,274 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:55:48,274 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:55:48,274 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37011,1676087618428' joining acquired barrier for procedure 'ns:testMinorCompaction' on coordinator 2023-02-11 03:55:48,274 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6bb5381d[Count = 0] remaining members to acquire global barrier 2023-02-11 03:55:48,274 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'ns:testMinorCompaction' starting 'in-barrier' execution. 2023-02-11 03:55:48,274 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-11 03:55:48,275 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-11 03:55:48,275 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-11 03:55:48,275 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-11 03:55:48,275 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'ns:testMinorCompaction' received 'reached' from coordinator. 2023-02-11 03:55:48,275 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'ns:testMinorCompaction' locally completed 2023-02-11 03:55:48,275 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'ns:testMinorCompaction' completed for member 'jenkins-hbase4.apache.org,37011,1676087618428' in zk 2023-02-11 03:55:48,275 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:55:48,275 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 03:55:48,277 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'ns:testMinorCompaction' has notified controller of completion 2023-02-11 03:55:48,277 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:55:48,277 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:55:48,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:55:48,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:55:48,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:55:48,277 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'ns:testMinorCompaction' completed. 2023-02-11 03:55:48,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:55:48,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:55:48,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-11 03:55:48,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:55:48,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:55:48,279 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-11 03:55:48,279 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:55:48,279 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'ns:testMinorCompaction' member 'jenkins-hbase4.apache.org,37011,1676087618428': 2023-02-11 03:55:48,279 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37011,1676087618428' released barrier for procedure'ns:testMinorCompaction', counting down latch. Waiting for 0 more 2023-02-11 03:55:48,279 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'ns:testMinorCompaction' execution completed 2023-02-11 03:55:48,279 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 03:55:48,279 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 03:55:48,279 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:ns:testMinorCompaction 2023-02-11 03:55:48,279 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure ns:testMinorCompactionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 03:55:48,281 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-11 03:55:48,281 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-11 03:55:48,281 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-11 03:55:48,281 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:55:48,281 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:55:48,281 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-11 03:55:48,281 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:55:48,281 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-11 03:55:48,281 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:55:48,281 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:55:48,281 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:55:48,282 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:55:48,282 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-11 03:55:48,282 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-11 03:55:48,282 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:55:48,282 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-11 03:55:48,283 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:55:48,283 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:55:48,283 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:55:48,283 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-11 03:55:48,284 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:55:48,285 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:55:48,285 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:55:48,285 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-02-11 03:55:48,285 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:55:48,285 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:55:48,285 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:55:48,285 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-02-11 03:55:48,285 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'ns:testMinorCompaction' 2023-02-11 03:55:48,285 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 03:55:48,285 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:55:48,285 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:55:48,285 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-11 03:55:48,285 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-11 03:55:48,285 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-11 03:55:48,286 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : ns:testMinorCompaction'' to complete. (max 300000 ms per retry) 2023-02-11 03:55:48,286 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 03:55:48,286 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:55:48,286 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:55:48,386 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 03:55:48,386 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 03:55:48,397 DEBUG [Listener at localhost/46205] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-02-11 03:55:48,406 DEBUG [Listener at localhost/46205] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-02-11 03:55:48,416 DEBUG [Listener at localhost/46205] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-02-11 03:55:48,416 DEBUG [Listener at localhost/46205] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-02-11 03:55:48,444 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 03:55:48,445 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] procedure.ProcedureCoordinator(143): Procedure ns:testMinorCompaction was in running list but was completed. Accepting new attempt. 2023-02-11 03:55:48,445 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] procedure.ProcedureCoordinator(165): Submitting procedure ns:testMinorCompaction 2023-02-11 03:55:48,446 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'ns:testMinorCompaction' 2023-02-11 03:55:48,446 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:55:48,446 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'ns:testMinorCompaction' starting 'acquire' 2023-02-11 03:55:48,446 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'ns:testMinorCompaction', kicking off acquire phase on members. 2023-02-11 03:55:48,446 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-11 03:55:48,446 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-02-11 03:55:48,448 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:55:48,448 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:55:48,448 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:55:48,448 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:55:48,448 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:55:48,448 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 03:55:48,448 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-02-11 03:55:48,449 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-11 03:55:48,449 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 03:55:48,449 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-02-11 03:55:48,449 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for ns:testMinorCompaction 2023-02-11 03:55:48,449 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc ns:testMinorCompaction is still present, removing 2023-02-11 03:55:48,449 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:ns:testMinorCompaction 2023-02-11 03:55:48,449 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'ns:testMinorCompaction' with timeout 60000ms 2023-02-11 03:55:48,449 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:55:48,449 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'ns:testMinorCompaction' starting 'acquire' stage 2023-02-11 03:55:48,450 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,,1676087739490.6938f3b95bbc15aff8fb4a9d24d9aaf5. 2023-02-11 03:55:48,450 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,1,1676087739490.3fa91eb317d63eaff7b1021985f3ad00. 2023-02-11 03:55:48,450 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,,1676087739490.6938f3b95bbc15aff8fb4a9d24d9aaf5. started... 2023-02-11 03:55:48,450 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,1,1676087739490.3fa91eb317d63eaff7b1021985f3ad00. started... 2023-02-11 03:55:48,450 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 6938f3b95bbc15aff8fb4a9d24d9aaf5 2/2 column families, dataSize=138 B heapSize=944 B 2023-02-11 03:55:48,450 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-1] regionserver.HRegion(2744): Flushing 3fa91eb317d63eaff7b1021985f3ad00 2/2 column families, dataSize=138 B heapSize=944 B 2023-02-11 03:55:48,450 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-02-11 03:55:48,450 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 03:55:48,450 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,2,1676087739490.08cc9e8540da3c04f9eb90b5ad80782a. 2023-02-11 03:55:48,450 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,2,1676087739490.08cc9e8540da3c04f9eb90b5ad80782a. started... 2023-02-11 03:55:48,450 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-2] regionserver.HRegion(2744): Flushing 08cc9e8540da3c04f9eb90b5ad80782a 2/2 column families, dataSize=138 B heapSize=944 B 2023-02-11 03:55:49,270 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/3fa91eb317d63eaff7b1021985f3ad00/.tmp/family1/f0a40290dce346538edec3b3aec69cc6 2023-02-11 03:55:49,273 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/6938f3b95bbc15aff8fb4a9d24d9aaf5/.tmp/family1/f6aa090fff754b0aad426c903712cc10 2023-02-11 03:55:49,274 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/08cc9e8540da3c04f9eb90b5ad80782a/.tmp/family1/233def8ce70643b0ac0b203d0efefa1d 2023-02-11 03:55:49,275 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for f0a40290dce346538edec3b3aec69cc6 2023-02-11 03:55:49,277 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for f6aa090fff754b0aad426c903712cc10 2023-02-11 03:55:49,278 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 233def8ce70643b0ac0b203d0efefa1d 2023-02-11 03:55:49,692 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/3fa91eb317d63eaff7b1021985f3ad00/.tmp/family2/0ec24d9d0042400d9a48f0b12db648d0 2023-02-11 03:55:49,696 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 0ec24d9d0042400d9a48f0b12db648d0 2023-02-11 03:55:49,697 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/3fa91eb317d63eaff7b1021985f3ad00/.tmp/family1/f0a40290dce346538edec3b3aec69cc6 as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/3fa91eb317d63eaff7b1021985f3ad00/family1/f0a40290dce346538edec3b3aec69cc6 2023-02-11 03:55:49,700 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for f0a40290dce346538edec3b3aec69cc6 2023-02-11 03:55:49,700 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/3fa91eb317d63eaff7b1021985f3ad00/family1/f0a40290dce346538edec3b3aec69cc6, entries=3, sequenceid=23, filesize=5.0 K 2023-02-11 03:55:49,701 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/3fa91eb317d63eaff7b1021985f3ad00/.tmp/family2/0ec24d9d0042400d9a48f0b12db648d0 as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/3fa91eb317d63eaff7b1021985f3ad00/family2/0ec24d9d0042400d9a48f0b12db648d0 2023-02-11 03:55:49,703 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 0ec24d9d0042400d9a48f0b12db648d0 2023-02-11 03:55:49,703 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/3fa91eb317d63eaff7b1021985f3ad00/family2/0ec24d9d0042400d9a48f0b12db648d0, entries=1, sequenceid=23, filesize=5.0 K 2023-02-11 03:55:49,704 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-1] regionserver.HRegion(2947): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for 3fa91eb317d63eaff7b1021985f3ad00 in 1254ms, sequenceid=23, compaction requested=true 2023-02-11 03:55:49,704 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-1] regionserver.HRegion(2445): Flush status journal for 3fa91eb317d63eaff7b1021985f3ad00: 2023-02-11 03:55:49,704 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,1,1676087739490.3fa91eb317d63eaff7b1021985f3ad00. 2023-02-11 03:55:49,704 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-02-11 03:55:50,095 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/6938f3b95bbc15aff8fb4a9d24d9aaf5/.tmp/family2/50d4ff5c808e43f48d8505636e5dc64f 2023-02-11 03:55:50,096 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/08cc9e8540da3c04f9eb90b5ad80782a/.tmp/family2/a2ea90f611824324863856f104dfa5c8 2023-02-11 03:55:50,099 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for a2ea90f611824324863856f104dfa5c8 2023-02-11 03:55:50,099 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 50d4ff5c808e43f48d8505636e5dc64f 2023-02-11 03:55:50,100 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/08cc9e8540da3c04f9eb90b5ad80782a/.tmp/family1/233def8ce70643b0ac0b203d0efefa1d as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/08cc9e8540da3c04f9eb90b5ad80782a/family1/233def8ce70643b0ac0b203d0efefa1d 2023-02-11 03:55:50,100 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/6938f3b95bbc15aff8fb4a9d24d9aaf5/.tmp/family1/f6aa090fff754b0aad426c903712cc10 as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/6938f3b95bbc15aff8fb4a9d24d9aaf5/family1/f6aa090fff754b0aad426c903712cc10 2023-02-11 03:55:50,104 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 233def8ce70643b0ac0b203d0efefa1d 2023-02-11 03:55:50,104 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for f6aa090fff754b0aad426c903712cc10 2023-02-11 03:55:50,104 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/08cc9e8540da3c04f9eb90b5ad80782a/family1/233def8ce70643b0ac0b203d0efefa1d, entries=3, sequenceid=23, filesize=5.0 K 2023-02-11 03:55:50,104 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/6938f3b95bbc15aff8fb4a9d24d9aaf5/family1/f6aa090fff754b0aad426c903712cc10, entries=3, sequenceid=23, filesize=5.0 K 2023-02-11 03:55:50,104 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/08cc9e8540da3c04f9eb90b5ad80782a/.tmp/family2/a2ea90f611824324863856f104dfa5c8 as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/08cc9e8540da3c04f9eb90b5ad80782a/family2/a2ea90f611824324863856f104dfa5c8 2023-02-11 03:55:50,104 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/6938f3b95bbc15aff8fb4a9d24d9aaf5/.tmp/family2/50d4ff5c808e43f48d8505636e5dc64f as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/6938f3b95bbc15aff8fb4a9d24d9aaf5/family2/50d4ff5c808e43f48d8505636e5dc64f 2023-02-11 03:55:50,109 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for a2ea90f611824324863856f104dfa5c8 2023-02-11 03:55:50,109 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/08cc9e8540da3c04f9eb90b5ad80782a/family2/a2ea90f611824324863856f104dfa5c8, entries=1, sequenceid=23, filesize=5.0 K 2023-02-11 03:55:50,109 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 50d4ff5c808e43f48d8505636e5dc64f 2023-02-11 03:55:50,110 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/6938f3b95bbc15aff8fb4a9d24d9aaf5/family2/50d4ff5c808e43f48d8505636e5dc64f, entries=1, sequenceid=23, filesize=5.0 K 2023-02-11 03:55:50,110 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-2] regionserver.HRegion(2947): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for 08cc9e8540da3c04f9eb90b5ad80782a in 1660ms, sequenceid=23, compaction requested=true 2023-02-11 03:55:50,110 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-2] regionserver.HRegion(2445): Flush status journal for 08cc9e8540da3c04f9eb90b5ad80782a: 2023-02-11 03:55:50,110 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,2,1676087739490.08cc9e8540da3c04f9eb90b5ad80782a. 2023-02-11 03:55:50,110 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-02-11 03:55:50,110 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for 6938f3b95bbc15aff8fb4a9d24d9aaf5 in 1660ms, sequenceid=23, compaction requested=true 2023-02-11 03:55:50,110 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 6938f3b95bbc15aff8fb4a9d24d9aaf5: 2023-02-11 03:55:50,110 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,,1676087739490.6938f3b95bbc15aff8fb4a9d24d9aaf5. 2023-02-11 03:55:50,110 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-02-11 03:55:50,110 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-02-11 03:55:50,110 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:55:50,110 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'ns:testMinorCompaction' locally acquired 2023-02-11 03:55:50,110 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37011,1676087618428' joining acquired barrier for procedure (ns:testMinorCompaction) in zk 2023-02-11 03:55:50,112 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:55:50,112 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-11 03:55:50,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:55:50,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:55:50,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:55:50,112 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-11 03:55:50,112 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'ns:testMinorCompaction' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 03:55:50,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:55:50,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:55:50,113 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-11 03:55:50,113 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:55:50,113 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:55:50,113 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37011,1676087618428' joining acquired barrier for procedure 'ns:testMinorCompaction' on coordinator 2023-02-11 03:55:50,113 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@62cb604c[Count = 0] remaining members to acquire global barrier 2023-02-11 03:55:50,113 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'ns:testMinorCompaction' starting 'in-barrier' execution. 2023-02-11 03:55:50,113 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-11 03:55:50,114 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-11 03:55:50,114 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-11 03:55:50,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-11 03:55:50,114 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'ns:testMinorCompaction' received 'reached' from coordinator. 2023-02-11 03:55:50,114 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'ns:testMinorCompaction' locally completed 2023-02-11 03:55:50,114 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'ns:testMinorCompaction' completed for member 'jenkins-hbase4.apache.org,37011,1676087618428' in zk 2023-02-11 03:55:50,115 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:55:50,115 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 03:55:50,116 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:55:50,116 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'ns:testMinorCompaction' has notified controller of completion 2023-02-11 03:55:50,116 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:55:50,116 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:55:50,116 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:55:50,116 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:55:50,116 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'ns:testMinorCompaction' completed. 2023-02-11 03:55:50,116 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:55:50,116 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:55:50,116 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-11 03:55:50,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:55:50,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:55:50,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-11 03:55:50,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:55:50,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'ns:testMinorCompaction' member 'jenkins-hbase4.apache.org,37011,1676087618428': 2023-02-11 03:55:50,118 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37011,1676087618428' released barrier for procedure'ns:testMinorCompaction', counting down latch. Waiting for 0 more 2023-02-11 03:55:50,118 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'ns:testMinorCompaction' execution completed 2023-02-11 03:55:50,118 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 03:55:50,118 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 03:55:50,118 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:ns:testMinorCompaction 2023-02-11 03:55:50,118 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure ns:testMinorCompactionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 03:55:50,123 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-11 03:55:50,123 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-11 03:55:50,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-11 03:55:50,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:55:50,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:55:50,123 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-11 03:55:50,123 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:55:50,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-11 03:55:50,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:55:50,123 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:55:50,123 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:55:50,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:55:50,124 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-11 03:55:50,124 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-11 03:55:50,124 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:55:50,124 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-11 03:55:50,124 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:55:50,124 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:55:50,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:55:50,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-11 03:55:50,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:55:50,126 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:55:50,126 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-02-11 03:55:50,126 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:55:50,126 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-02-11 03:55:50,126 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:55:50,126 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:55:50,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:55:50,126 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'ns:testMinorCompaction' 2023-02-11 03:55:50,127 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 03:55:50,126 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:55:50,126 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:55:50,126 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-11 03:55:50,127 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-11 03:55:50,127 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-11 03:55:50,127 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : ns:testMinorCompaction'' to complete. (max 300000 ms per retry) 2023-02-11 03:55:50,127 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:55:50,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:55:50,127 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 03:55:50,227 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 03:55:50,228 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39921] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 03:55:50,228 DEBUG [Listener at localhost/46205] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-02-11 03:55:50,228 DEBUG [Listener at localhost/46205] regionserver.HStore(1912): 6938f3b95bbc15aff8fb4a9d24d9aaf5/family1 is initiating major compaction (all files) 2023-02-11 03:55:50,229 INFO [Listener at localhost/46205] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-11 03:55:50,229 INFO [Listener at localhost/46205] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-11 03:55:50,229 INFO [Listener at localhost/46205] regionserver.HRegion(2258): Starting compaction of 6938f3b95bbc15aff8fb4a9d24d9aaf5/family1 in ns:testMinorCompaction,,1676087739490.6938f3b95bbc15aff8fb4a9d24d9aaf5. 2023-02-11 03:55:50,229 INFO [Listener at localhost/46205] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/6938f3b95bbc15aff8fb4a9d24d9aaf5/family1/cf56fda2084b4736b3389b67629c9b2a, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/6938f3b95bbc15aff8fb4a9d24d9aaf5/family1/3f86c29984af496d8a85fdfd4185e520, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/6938f3b95bbc15aff8fb4a9d24d9aaf5/family1/74afb6ba1d1348a09ccd3bbccc4f4536, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/6938f3b95bbc15aff8fb4a9d24d9aaf5/family1/452e95bc02544b499b6dcea6dae339ec, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/6938f3b95bbc15aff8fb4a9d24d9aaf5/family1/f6aa090fff754b0aad426c903712cc10] into tmpdir=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/6938f3b95bbc15aff8fb4a9d24d9aaf5/.tmp, totalSize=26.6 K 2023-02-11 03:55:50,229 DEBUG [Listener at localhost/46205] compactions.Compactor(207): Compacting cf56fda2084b4736b3389b67629c9b2a, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1676087741604 2023-02-11 03:55:50,230 DEBUG [Listener at localhost/46205] compactions.Compactor(207): Compacting 3f86c29984af496d8a85fdfd4185e520, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1676087743389 2023-02-11 03:55:50,230 DEBUG [Listener at localhost/46205] compactions.Compactor(207): Compacting 74afb6ba1d1348a09ccd3bbccc4f4536, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1676087744824 2023-02-11 03:55:50,230 DEBUG [Listener at localhost/46205] compactions.Compactor(207): Compacting 452e95bc02544b499b6dcea6dae339ec, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1676087746607 2023-02-11 03:55:50,230 DEBUG [Listener at localhost/46205] compactions.Compactor(207): Compacting f6aa090fff754b0aad426c903712cc10, keycount=3, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-02-11 03:55:50,239 INFO [Listener at localhost/46205] throttle.PressureAwareThroughputController(145): 6938f3b95bbc15aff8fb4a9d24d9aaf5#family1#compaction#149 average throughput is 0.55 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-11 03:55:50,246 INFO [Listener at localhost/46205] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e202302111760d094e96349f196c9c6df7e3abb5c_del 2023-02-11 03:55:50,246 INFO [Listener at localhost/46205] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202302111760d094e96349f196c9c6df7e3abb5c_del to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/d41d8cd98f00b204e9800998ecf8427e202302111760d094e96349f196c9c6df7e3abb5c_del 2023-02-11 03:55:51,069 INFO [Listener at localhost/46205] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for e2c77097805c4dbba6e7ce6ccd08e563 2023-02-11 03:55:51,069 DEBUG [Listener at localhost/46205] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/6938f3b95bbc15aff8fb4a9d24d9aaf5/.tmp/family1/e2c77097805c4dbba6e7ce6ccd08e563 as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/6938f3b95bbc15aff8fb4a9d24d9aaf5/family1/e2c77097805c4dbba6e7ce6ccd08e563 2023-02-11 03:55:51,072 INFO [Listener at localhost/46205] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for e2c77097805c4dbba6e7ce6ccd08e563 2023-02-11 03:55:51,073 INFO [Listener at localhost/46205] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 6938f3b95bbc15aff8fb4a9d24d9aaf5/family1 of 6938f3b95bbc15aff8fb4a9d24d9aaf5 into e2c77097805c4dbba6e7ce6ccd08e563(size=6.8 K), total size for store is 6.8 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-11 03:55:51,073 DEBUG [Listener at localhost/46205] regionserver.HRegion(2288): Compaction status journal for 6938f3b95bbc15aff8fb4a9d24d9aaf5: 2023-02-11 03:55:51,073 DEBUG [Listener at localhost/46205] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-02-11 03:55:51,073 DEBUG [Listener at localhost/46205] regionserver.HStore(1912): 6938f3b95bbc15aff8fb4a9d24d9aaf5/family2 is initiating major compaction (all files) 2023-02-11 03:55:51,073 INFO [Listener at localhost/46205] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-11 03:55:51,074 INFO [Listener at localhost/46205] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-11 03:55:51,074 INFO [Listener at localhost/46205] regionserver.HRegion(2258): Starting compaction of 6938f3b95bbc15aff8fb4a9d24d9aaf5/family2 in ns:testMinorCompaction,,1676087739490.6938f3b95bbc15aff8fb4a9d24d9aaf5. 2023-02-11 03:55:51,074 INFO [Listener at localhost/46205] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/6938f3b95bbc15aff8fb4a9d24d9aaf5/family2/9ec3c577685c478c85883a8dfde7d662, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/6938f3b95bbc15aff8fb4a9d24d9aaf5/family2/9c5820d8e743439fab3b9790d24f84ce, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/6938f3b95bbc15aff8fb4a9d24d9aaf5/family2/d981d090bce74425ba2cbbcdded59840, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/6938f3b95bbc15aff8fb4a9d24d9aaf5/family2/208d33478a60482482546463e56b36ce, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/6938f3b95bbc15aff8fb4a9d24d9aaf5/family2/50d4ff5c808e43f48d8505636e5dc64f] into tmpdir=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/6938f3b95bbc15aff8fb4a9d24d9aaf5/.tmp, totalSize=25.3 K 2023-02-11 03:55:51,075 DEBUG [Listener at localhost/46205] compactions.Compactor(207): Compacting 9ec3c577685c478c85883a8dfde7d662, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1676087741604 2023-02-11 03:55:51,075 DEBUG [Listener at localhost/46205] compactions.Compactor(207): Compacting 9c5820d8e743439fab3b9790d24f84ce, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1676087743389 2023-02-11 03:55:51,075 DEBUG [Listener at localhost/46205] compactions.Compactor(207): Compacting d981d090bce74425ba2cbbcdded59840, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1676087744824 2023-02-11 03:55:51,076 DEBUG [Listener at localhost/46205] compactions.Compactor(207): Compacting 208d33478a60482482546463e56b36ce, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1676087746607 2023-02-11 03:55:51,076 DEBUG [Listener at localhost/46205] compactions.Compactor(207): Compacting 50d4ff5c808e43f48d8505636e5dc64f, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-02-11 03:55:51,083 INFO [Listener at localhost/46205] throttle.PressureAwareThroughputController(145): 6938f3b95bbc15aff8fb4a9d24d9aaf5#family2#compaction#150 average throughput is 0.51 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-11 03:55:51,149 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/6938f3b95bbc15aff8fb4a9d24d9aaf5/family1/cf56fda2084b4736b3389b67629c9b2a, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/6938f3b95bbc15aff8fb4a9d24d9aaf5/family1/3f86c29984af496d8a85fdfd4185e520, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/6938f3b95bbc15aff8fb4a9d24d9aaf5/family1/74afb6ba1d1348a09ccd3bbccc4f4536, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/6938f3b95bbc15aff8fb4a9d24d9aaf5/family1/452e95bc02544b499b6dcea6dae339ec, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/6938f3b95bbc15aff8fb4a9d24d9aaf5/family1/f6aa090fff754b0aad426c903712cc10] to archive 2023-02-11 03:55:51,150 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-11 03:55:51,151 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/6938f3b95bbc15aff8fb4a9d24d9aaf5/family1/cf56fda2084b4736b3389b67629c9b2a to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/ns/testMinorCompaction/6938f3b95bbc15aff8fb4a9d24d9aaf5/family1/cf56fda2084b4736b3389b67629c9b2a 2023-02-11 03:55:51,152 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/6938f3b95bbc15aff8fb4a9d24d9aaf5/family1/3f86c29984af496d8a85fdfd4185e520 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/ns/testMinorCompaction/6938f3b95bbc15aff8fb4a9d24d9aaf5/family1/3f86c29984af496d8a85fdfd4185e520 2023-02-11 03:55:51,153 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/6938f3b95bbc15aff8fb4a9d24d9aaf5/family1/74afb6ba1d1348a09ccd3bbccc4f4536 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/ns/testMinorCompaction/6938f3b95bbc15aff8fb4a9d24d9aaf5/family1/74afb6ba1d1348a09ccd3bbccc4f4536 2023-02-11 03:55:51,154 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/6938f3b95bbc15aff8fb4a9d24d9aaf5/family1/452e95bc02544b499b6dcea6dae339ec to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/ns/testMinorCompaction/6938f3b95bbc15aff8fb4a9d24d9aaf5/family1/452e95bc02544b499b6dcea6dae339ec 2023-02-11 03:55:51,155 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/6938f3b95bbc15aff8fb4a9d24d9aaf5/family1/f6aa090fff754b0aad426c903712cc10 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/ns/testMinorCompaction/6938f3b95bbc15aff8fb4a9d24d9aaf5/family1/f6aa090fff754b0aad426c903712cc10 2023-02-11 03:55:51,491 INFO [Listener at localhost/46205] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e202302119bc51342182440408e2af340e2c58f8f_del 2023-02-11 03:55:51,491 INFO [Listener at localhost/46205] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202302119bc51342182440408e2af340e2c58f8f_del to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/d41d8cd98f00b204e9800998ecf8427e202302119bc51342182440408e2af340e2c58f8f_del 2023-02-11 03:55:52,303 INFO [Listener at localhost/46205] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for bd16b0c6694e4130b0c0756ca336ff84 2023-02-11 03:55:52,304 DEBUG [Listener at localhost/46205] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/6938f3b95bbc15aff8fb4a9d24d9aaf5/.tmp/family2/bd16b0c6694e4130b0c0756ca336ff84 as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/6938f3b95bbc15aff8fb4a9d24d9aaf5/family2/bd16b0c6694e4130b0c0756ca336ff84 2023-02-11 03:55:52,307 INFO [Listener at localhost/46205] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for bd16b0c6694e4130b0c0756ca336ff84 2023-02-11 03:55:52,308 INFO [Listener at localhost/46205] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 6938f3b95bbc15aff8fb4a9d24d9aaf5/family2 of 6938f3b95bbc15aff8fb4a9d24d9aaf5 into bd16b0c6694e4130b0c0756ca336ff84(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-02-11 03:55:52,308 DEBUG [Listener at localhost/46205] regionserver.HRegion(2288): Compaction status journal for 6938f3b95bbc15aff8fb4a9d24d9aaf5: 2023-02-11 03:55:52,308 DEBUG [Listener at localhost/46205] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-02-11 03:55:52,308 DEBUG [Listener at localhost/46205] regionserver.HStore(1912): 3fa91eb317d63eaff7b1021985f3ad00/family1 is initiating major compaction (all files) 2023-02-11 03:55:52,308 INFO [Listener at localhost/46205] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-11 03:55:52,308 INFO [Listener at localhost/46205] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-11 03:55:52,308 INFO [Listener at localhost/46205] regionserver.HRegion(2258): Starting compaction of 3fa91eb317d63eaff7b1021985f3ad00/family1 in ns:testMinorCompaction,1,1676087739490.3fa91eb317d63eaff7b1021985f3ad00. 2023-02-11 03:55:52,308 INFO [Listener at localhost/46205] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/3fa91eb317d63eaff7b1021985f3ad00/family1/4da5a5dcbded4cb9ad85833159091832, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/3fa91eb317d63eaff7b1021985f3ad00/family1/971239e058184febb54e467f537ea248, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/3fa91eb317d63eaff7b1021985f3ad00/family1/2ad4cd5d0be44732a4a22dd4489bde40, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/3fa91eb317d63eaff7b1021985f3ad00/family1/4ae66d741c9441a180df4ce64e5983a1, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/3fa91eb317d63eaff7b1021985f3ad00/family1/f0a40290dce346538edec3b3aec69cc6] into tmpdir=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/3fa91eb317d63eaff7b1021985f3ad00/.tmp, totalSize=26.6 K 2023-02-11 03:55:52,309 DEBUG [Listener at localhost/46205] compactions.Compactor(207): Compacting 4da5a5dcbded4cb9ad85833159091832, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1676087741604 2023-02-11 03:55:52,309 DEBUG [Listener at localhost/46205] compactions.Compactor(207): Compacting 971239e058184febb54e467f537ea248, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1676087743389 2023-02-11 03:55:52,309 DEBUG [Listener at localhost/46205] compactions.Compactor(207): Compacting 2ad4cd5d0be44732a4a22dd4489bde40, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1676087744825 2023-02-11 03:55:52,309 DEBUG [Listener at localhost/46205] compactions.Compactor(207): Compacting 4ae66d741c9441a180df4ce64e5983a1, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1676087746607 2023-02-11 03:55:52,310 DEBUG [Listener at localhost/46205] compactions.Compactor(207): Compacting f0a40290dce346538edec3b3aec69cc6, keycount=3, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-02-11 03:55:52,317 INFO [Listener at localhost/46205] throttle.PressureAwareThroughputController(145): 3fa91eb317d63eaff7b1021985f3ad00#family1#compaction#151 average throughput is 0.83 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-11 03:55:52,350 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/6938f3b95bbc15aff8fb4a9d24d9aaf5/family2/9ec3c577685c478c85883a8dfde7d662, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/6938f3b95bbc15aff8fb4a9d24d9aaf5/family2/9c5820d8e743439fab3b9790d24f84ce, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/6938f3b95bbc15aff8fb4a9d24d9aaf5/family2/d981d090bce74425ba2cbbcdded59840, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/6938f3b95bbc15aff8fb4a9d24d9aaf5/family2/208d33478a60482482546463e56b36ce, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/6938f3b95bbc15aff8fb4a9d24d9aaf5/family2/50d4ff5c808e43f48d8505636e5dc64f] to archive 2023-02-11 03:55:52,351 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-11 03:55:52,353 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/6938f3b95bbc15aff8fb4a9d24d9aaf5/family2/9ec3c577685c478c85883a8dfde7d662 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/ns/testMinorCompaction/6938f3b95bbc15aff8fb4a9d24d9aaf5/family2/9ec3c577685c478c85883a8dfde7d662 2023-02-11 03:55:52,354 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/6938f3b95bbc15aff8fb4a9d24d9aaf5/family2/9c5820d8e743439fab3b9790d24f84ce to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/ns/testMinorCompaction/6938f3b95bbc15aff8fb4a9d24d9aaf5/family2/9c5820d8e743439fab3b9790d24f84ce 2023-02-11 03:55:52,354 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/6938f3b95bbc15aff8fb4a9d24d9aaf5/family2/d981d090bce74425ba2cbbcdded59840 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/ns/testMinorCompaction/6938f3b95bbc15aff8fb4a9d24d9aaf5/family2/d981d090bce74425ba2cbbcdded59840 2023-02-11 03:55:52,355 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/6938f3b95bbc15aff8fb4a9d24d9aaf5/family2/208d33478a60482482546463e56b36ce to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/ns/testMinorCompaction/6938f3b95bbc15aff8fb4a9d24d9aaf5/family2/208d33478a60482482546463e56b36ce 2023-02-11 03:55:52,356 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/6938f3b95bbc15aff8fb4a9d24d9aaf5/family2/50d4ff5c808e43f48d8505636e5dc64f to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/ns/testMinorCompaction/6938f3b95bbc15aff8fb4a9d24d9aaf5/family2/50d4ff5c808e43f48d8505636e5dc64f 2023-02-11 03:55:52,726 INFO [Listener at localhost/46205] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b20230211988c0389c6b44d1291d66a7b4acee814_del 2023-02-11 03:55:52,726 INFO [Listener at localhost/46205] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230211988c0389c6b44d1291d66a7b4acee814_del to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c4ca4238a0b923820dcc509a6f75849b20230211988c0389c6b44d1291d66a7b4acee814_del 2023-02-11 03:55:53,540 INFO [Listener at localhost/46205] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for a8c9ba8b5cff47299c7cbf1f719714a1 2023-02-11 03:55:53,541 DEBUG [Listener at localhost/46205] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/3fa91eb317d63eaff7b1021985f3ad00/.tmp/family1/a8c9ba8b5cff47299c7cbf1f719714a1 as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/3fa91eb317d63eaff7b1021985f3ad00/family1/a8c9ba8b5cff47299c7cbf1f719714a1 2023-02-11 03:55:53,545 INFO [Listener at localhost/46205] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for a8c9ba8b5cff47299c7cbf1f719714a1 2023-02-11 03:55:53,545 INFO [Listener at localhost/46205] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 3fa91eb317d63eaff7b1021985f3ad00/family1 of 3fa91eb317d63eaff7b1021985f3ad00 into a8c9ba8b5cff47299c7cbf1f719714a1(size=6.8 K), total size for store is 6.8 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-02-11 03:55:53,546 DEBUG [Listener at localhost/46205] regionserver.HRegion(2288): Compaction status journal for 3fa91eb317d63eaff7b1021985f3ad00: 2023-02-11 03:55:53,546 DEBUG [Listener at localhost/46205] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-02-11 03:55:53,546 DEBUG [Listener at localhost/46205] regionserver.HStore(1912): 3fa91eb317d63eaff7b1021985f3ad00/family2 is initiating major compaction (all files) 2023-02-11 03:55:53,546 INFO [Listener at localhost/46205] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-11 03:55:53,546 INFO [Listener at localhost/46205] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-11 03:55:53,547 INFO [Listener at localhost/46205] regionserver.HRegion(2258): Starting compaction of 3fa91eb317d63eaff7b1021985f3ad00/family2 in ns:testMinorCompaction,1,1676087739490.3fa91eb317d63eaff7b1021985f3ad00. 2023-02-11 03:55:53,547 INFO [Listener at localhost/46205] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/3fa91eb317d63eaff7b1021985f3ad00/family2/f3d9cf4338244975a22488a625861fd5, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/3fa91eb317d63eaff7b1021985f3ad00/family2/7750003a52504750aa8012b5abcdd4ee, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/3fa91eb317d63eaff7b1021985f3ad00/family2/5bc433a5b8d7447bb75bfea1ccc3c50c, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/3fa91eb317d63eaff7b1021985f3ad00/family2/c4be4f3faf124a13ab03497afaac436c, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/3fa91eb317d63eaff7b1021985f3ad00/family2/0ec24d9d0042400d9a48f0b12db648d0] into tmpdir=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/3fa91eb317d63eaff7b1021985f3ad00/.tmp, totalSize=25.3 K 2023-02-11 03:55:53,547 DEBUG [Listener at localhost/46205] compactions.Compactor(207): Compacting f3d9cf4338244975a22488a625861fd5, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1676087741604 2023-02-11 03:55:53,548 DEBUG [Listener at localhost/46205] compactions.Compactor(207): Compacting 7750003a52504750aa8012b5abcdd4ee, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1676087743389 2023-02-11 03:55:53,548 DEBUG [Listener at localhost/46205] compactions.Compactor(207): Compacting 5bc433a5b8d7447bb75bfea1ccc3c50c, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1676087744825 2023-02-11 03:55:53,548 DEBUG [Listener at localhost/46205] compactions.Compactor(207): Compacting c4be4f3faf124a13ab03497afaac436c, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1676087746607 2023-02-11 03:55:53,549 DEBUG [Listener at localhost/46205] compactions.Compactor(207): Compacting 0ec24d9d0042400d9a48f0b12db648d0, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-02-11 03:55:53,549 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/3fa91eb317d63eaff7b1021985f3ad00/family1/4da5a5dcbded4cb9ad85833159091832, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/3fa91eb317d63eaff7b1021985f3ad00/family1/971239e058184febb54e467f537ea248, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/3fa91eb317d63eaff7b1021985f3ad00/family1/2ad4cd5d0be44732a4a22dd4489bde40, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/3fa91eb317d63eaff7b1021985f3ad00/family1/4ae66d741c9441a180df4ce64e5983a1, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/3fa91eb317d63eaff7b1021985f3ad00/family1/f0a40290dce346538edec3b3aec69cc6] to archive 2023-02-11 03:55:53,550 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-11 03:55:53,552 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/3fa91eb317d63eaff7b1021985f3ad00/family1/4da5a5dcbded4cb9ad85833159091832 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/ns/testMinorCompaction/3fa91eb317d63eaff7b1021985f3ad00/family1/4da5a5dcbded4cb9ad85833159091832 2023-02-11 03:55:53,553 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/3fa91eb317d63eaff7b1021985f3ad00/family1/971239e058184febb54e467f537ea248 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/ns/testMinorCompaction/3fa91eb317d63eaff7b1021985f3ad00/family1/971239e058184febb54e467f537ea248 2023-02-11 03:55:53,555 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/3fa91eb317d63eaff7b1021985f3ad00/family1/2ad4cd5d0be44732a4a22dd4489bde40 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/ns/testMinorCompaction/3fa91eb317d63eaff7b1021985f3ad00/family1/2ad4cd5d0be44732a4a22dd4489bde40 2023-02-11 03:55:53,556 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/3fa91eb317d63eaff7b1021985f3ad00/family1/4ae66d741c9441a180df4ce64e5983a1 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/ns/testMinorCompaction/3fa91eb317d63eaff7b1021985f3ad00/family1/4ae66d741c9441a180df4ce64e5983a1 2023-02-11 03:55:53,558 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/3fa91eb317d63eaff7b1021985f3ad00/family1/f0a40290dce346538edec3b3aec69cc6 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/ns/testMinorCompaction/3fa91eb317d63eaff7b1021985f3ad00/family1/f0a40290dce346538edec3b3aec69cc6 2023-02-11 03:55:53,563 INFO [Listener at localhost/46205] throttle.PressureAwareThroughputController(145): 3fa91eb317d63eaff7b1021985f3ad00#family2#compaction#152 average throughput is 0.51 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-11 03:55:53,970 INFO [Listener at localhost/46205] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b20230211d90aec85a228412a84faa11e9306edd5_del 2023-02-11 03:55:53,970 INFO [Listener at localhost/46205] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230211d90aec85a228412a84faa11e9306edd5_del to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c4ca4238a0b923820dcc509a6f75849b20230211d90aec85a228412a84faa11e9306edd5_del 2023-02-11 03:55:54,784 INFO [Listener at localhost/46205] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c8f0a95acdf149d1b60efcdc03fa7024 2023-02-11 03:55:54,784 DEBUG [Listener at localhost/46205] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/3fa91eb317d63eaff7b1021985f3ad00/.tmp/family2/c8f0a95acdf149d1b60efcdc03fa7024 as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/3fa91eb317d63eaff7b1021985f3ad00/family2/c8f0a95acdf149d1b60efcdc03fa7024 2023-02-11 03:55:54,788 INFO [Listener at localhost/46205] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c8f0a95acdf149d1b60efcdc03fa7024 2023-02-11 03:55:54,788 INFO [Listener at localhost/46205] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 3fa91eb317d63eaff7b1021985f3ad00/family2 of 3fa91eb317d63eaff7b1021985f3ad00 into c8f0a95acdf149d1b60efcdc03fa7024(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-02-11 03:55:54,789 DEBUG [Listener at localhost/46205] regionserver.HRegion(2288): Compaction status journal for 3fa91eb317d63eaff7b1021985f3ad00: 2023-02-11 03:55:54,789 DEBUG [Listener at localhost/46205] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-02-11 03:55:54,789 DEBUG [Listener at localhost/46205] regionserver.HStore(1912): 08cc9e8540da3c04f9eb90b5ad80782a/family1 is initiating major compaction (all files) 2023-02-11 03:55:54,789 INFO [Listener at localhost/46205] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-11 03:55:54,789 INFO [Listener at localhost/46205] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-11 03:55:54,789 INFO [Listener at localhost/46205] regionserver.HRegion(2258): Starting compaction of 08cc9e8540da3c04f9eb90b5ad80782a/family1 in ns:testMinorCompaction,2,1676087739490.08cc9e8540da3c04f9eb90b5ad80782a. 2023-02-11 03:55:54,789 INFO [Listener at localhost/46205] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/08cc9e8540da3c04f9eb90b5ad80782a/family1/1965537a06594723af60377479103e74, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/08cc9e8540da3c04f9eb90b5ad80782a/family1/42cb79fafc66413294674382953efdf2, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/08cc9e8540da3c04f9eb90b5ad80782a/family1/c44faad7b3744429af39663af5118d17, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/08cc9e8540da3c04f9eb90b5ad80782a/family1/e607cd8b09b34b92bd7f72d6940e698d, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/08cc9e8540da3c04f9eb90b5ad80782a/family1/233def8ce70643b0ac0b203d0efefa1d] into tmpdir=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/08cc9e8540da3c04f9eb90b5ad80782a/.tmp, totalSize=26.6 K 2023-02-11 03:55:54,789 DEBUG [Listener at localhost/46205] compactions.Compactor(207): Compacting 1965537a06594723af60377479103e74, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1676087741605 2023-02-11 03:55:54,790 DEBUG [Listener at localhost/46205] compactions.Compactor(207): Compacting 42cb79fafc66413294674382953efdf2, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1676087743389 2023-02-11 03:55:54,790 DEBUG [Listener at localhost/46205] compactions.Compactor(207): Compacting c44faad7b3744429af39663af5118d17, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1676087744825 2023-02-11 03:55:54,790 DEBUG [Listener at localhost/46205] compactions.Compactor(207): Compacting e607cd8b09b34b92bd7f72d6940e698d, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1676087746607 2023-02-11 03:55:54,791 DEBUG [Listener at localhost/46205] compactions.Compactor(207): Compacting 233def8ce70643b0ac0b203d0efefa1d, keycount=3, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-02-11 03:55:54,799 INFO [Listener at localhost/46205] throttle.PressureAwareThroughputController(145): 08cc9e8540da3c04f9eb90b5ad80782a#family1#compaction#153 average throughput is 1.65 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-11 03:55:54,849 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/3fa91eb317d63eaff7b1021985f3ad00/family2/f3d9cf4338244975a22488a625861fd5, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/3fa91eb317d63eaff7b1021985f3ad00/family2/7750003a52504750aa8012b5abcdd4ee, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/3fa91eb317d63eaff7b1021985f3ad00/family2/5bc433a5b8d7447bb75bfea1ccc3c50c, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/3fa91eb317d63eaff7b1021985f3ad00/family2/c4be4f3faf124a13ab03497afaac436c, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/3fa91eb317d63eaff7b1021985f3ad00/family2/0ec24d9d0042400d9a48f0b12db648d0] to archive 2023-02-11 03:55:54,850 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-11 03:55:54,852 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/3fa91eb317d63eaff7b1021985f3ad00/family2/f3d9cf4338244975a22488a625861fd5 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/ns/testMinorCompaction/3fa91eb317d63eaff7b1021985f3ad00/family2/f3d9cf4338244975a22488a625861fd5 2023-02-11 03:55:54,853 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/3fa91eb317d63eaff7b1021985f3ad00/family2/7750003a52504750aa8012b5abcdd4ee to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/ns/testMinorCompaction/3fa91eb317d63eaff7b1021985f3ad00/family2/7750003a52504750aa8012b5abcdd4ee 2023-02-11 03:55:54,854 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/3fa91eb317d63eaff7b1021985f3ad00/family2/5bc433a5b8d7447bb75bfea1ccc3c50c to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/ns/testMinorCompaction/3fa91eb317d63eaff7b1021985f3ad00/family2/5bc433a5b8d7447bb75bfea1ccc3c50c 2023-02-11 03:55:54,854 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/3fa91eb317d63eaff7b1021985f3ad00/family2/c4be4f3faf124a13ab03497afaac436c to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/ns/testMinorCompaction/3fa91eb317d63eaff7b1021985f3ad00/family2/c4be4f3faf124a13ab03497afaac436c 2023-02-11 03:55:54,855 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/3fa91eb317d63eaff7b1021985f3ad00/family2/0ec24d9d0042400d9a48f0b12db648d0 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/ns/testMinorCompaction/3fa91eb317d63eaff7b1021985f3ad00/family2/0ec24d9d0042400d9a48f0b12db648d0 2023-02-11 03:55:55,205 INFO [Listener at localhost/46205] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c202302111c6ff7965ebf424da0357ae03949d326_del 2023-02-11 03:55:55,206 INFO [Listener at localhost/46205] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202302111c6ff7965ebf424da0357ae03949d326_del to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c81e728d9d4c2f636f067f89cc14862c202302111c6ff7965ebf424da0357ae03949d326_del 2023-02-11 03:55:56,018 INFO [Listener at localhost/46205] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 62849d7659dc45a2bd2a85856988df57 2023-02-11 03:55:56,019 DEBUG [Listener at localhost/46205] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/08cc9e8540da3c04f9eb90b5ad80782a/.tmp/family1/62849d7659dc45a2bd2a85856988df57 as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/08cc9e8540da3c04f9eb90b5ad80782a/family1/62849d7659dc45a2bd2a85856988df57 2023-02-11 03:55:56,022 INFO [Listener at localhost/46205] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 62849d7659dc45a2bd2a85856988df57 2023-02-11 03:55:56,023 INFO [Listener at localhost/46205] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 08cc9e8540da3c04f9eb90b5ad80782a/family1 of 08cc9e8540da3c04f9eb90b5ad80782a into 62849d7659dc45a2bd2a85856988df57(size=6.8 K), total size for store is 6.8 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-02-11 03:55:56,023 DEBUG [Listener at localhost/46205] regionserver.HRegion(2288): Compaction status journal for 08cc9e8540da3c04f9eb90b5ad80782a: 2023-02-11 03:55:56,024 DEBUG [Listener at localhost/46205] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-02-11 03:55:56,024 DEBUG [Listener at localhost/46205] regionserver.HStore(1912): 08cc9e8540da3c04f9eb90b5ad80782a/family2 is initiating major compaction (all files) 2023-02-11 03:55:56,024 INFO [Listener at localhost/46205] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-11 03:55:56,024 INFO [Listener at localhost/46205] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-11 03:55:56,024 INFO [Listener at localhost/46205] regionserver.HRegion(2258): Starting compaction of 08cc9e8540da3c04f9eb90b5ad80782a/family2 in ns:testMinorCompaction,2,1676087739490.08cc9e8540da3c04f9eb90b5ad80782a. 2023-02-11 03:55:56,024 INFO [Listener at localhost/46205] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/08cc9e8540da3c04f9eb90b5ad80782a/family2/40cb6bf4cd204ced9ed8f8b62e7eb74b, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/08cc9e8540da3c04f9eb90b5ad80782a/family2/2ae65cf0e4db479c8f62d84e1541cfe0, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/08cc9e8540da3c04f9eb90b5ad80782a/family2/be792f70fa374c46a3ecacbbae329638, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/08cc9e8540da3c04f9eb90b5ad80782a/family2/fa32da4a61cf4a7794826f9b7f890709, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/08cc9e8540da3c04f9eb90b5ad80782a/family2/a2ea90f611824324863856f104dfa5c8] into tmpdir=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/08cc9e8540da3c04f9eb90b5ad80782a/.tmp, totalSize=25.3 K 2023-02-11 03:55:56,024 DEBUG [Listener at localhost/46205] compactions.Compactor(207): Compacting 40cb6bf4cd204ced9ed8f8b62e7eb74b, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1676087741605 2023-02-11 03:55:56,025 DEBUG [Listener at localhost/46205] compactions.Compactor(207): Compacting 2ae65cf0e4db479c8f62d84e1541cfe0, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1676087743389 2023-02-11 03:55:56,025 DEBUG [Listener at localhost/46205] compactions.Compactor(207): Compacting be792f70fa374c46a3ecacbbae329638, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1676087744825 2023-02-11 03:55:56,025 DEBUG [Listener at localhost/46205] compactions.Compactor(207): Compacting fa32da4a61cf4a7794826f9b7f890709, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1676087746607 2023-02-11 03:55:56,026 DEBUG [Listener at localhost/46205] compactions.Compactor(207): Compacting a2ea90f611824324863856f104dfa5c8, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-02-11 03:55:56,033 INFO [Listener at localhost/46205] throttle.PressureAwareThroughputController(145): 08cc9e8540da3c04f9eb90b5ad80782a#family2#compaction#154 average throughput is 0.51 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-11 03:55:56,049 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/08cc9e8540da3c04f9eb90b5ad80782a/family1/1965537a06594723af60377479103e74, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/08cc9e8540da3c04f9eb90b5ad80782a/family1/42cb79fafc66413294674382953efdf2, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/08cc9e8540da3c04f9eb90b5ad80782a/family1/c44faad7b3744429af39663af5118d17, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/08cc9e8540da3c04f9eb90b5ad80782a/family1/e607cd8b09b34b92bd7f72d6940e698d, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/08cc9e8540da3c04f9eb90b5ad80782a/family1/233def8ce70643b0ac0b203d0efefa1d] to archive 2023-02-11 03:55:56,050 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-11 03:55:56,051 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/08cc9e8540da3c04f9eb90b5ad80782a/family1/1965537a06594723af60377479103e74 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/ns/testMinorCompaction/08cc9e8540da3c04f9eb90b5ad80782a/family1/1965537a06594723af60377479103e74 2023-02-11 03:55:56,052 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/08cc9e8540da3c04f9eb90b5ad80782a/family1/42cb79fafc66413294674382953efdf2 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/ns/testMinorCompaction/08cc9e8540da3c04f9eb90b5ad80782a/family1/42cb79fafc66413294674382953efdf2 2023-02-11 03:55:56,053 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/08cc9e8540da3c04f9eb90b5ad80782a/family1/c44faad7b3744429af39663af5118d17 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/ns/testMinorCompaction/08cc9e8540da3c04f9eb90b5ad80782a/family1/c44faad7b3744429af39663af5118d17 2023-02-11 03:55:56,054 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/08cc9e8540da3c04f9eb90b5ad80782a/family1/e607cd8b09b34b92bd7f72d6940e698d to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/ns/testMinorCompaction/08cc9e8540da3c04f9eb90b5ad80782a/family1/e607cd8b09b34b92bd7f72d6940e698d 2023-02-11 03:55:56,055 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/08cc9e8540da3c04f9eb90b5ad80782a/family1/233def8ce70643b0ac0b203d0efefa1d to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/ns/testMinorCompaction/08cc9e8540da3c04f9eb90b5ad80782a/family1/233def8ce70643b0ac0b203d0efefa1d 2023-02-11 03:55:56,441 INFO [Listener at localhost/46205] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c202302117b265ea5a6254c4eaec290b4c7418302_del 2023-02-11 03:55:56,441 INFO [Listener at localhost/46205] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202302117b265ea5a6254c4eaec290b4c7418302_del to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c81e728d9d4c2f636f067f89cc14862c202302117b265ea5a6254c4eaec290b4c7418302_del 2023-02-11 03:55:57,254 INFO [Listener at localhost/46205] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 217312c2aebc436ab7d37237cf5bdefa 2023-02-11 03:55:57,255 DEBUG [Listener at localhost/46205] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/08cc9e8540da3c04f9eb90b5ad80782a/.tmp/family2/217312c2aebc436ab7d37237cf5bdefa as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/08cc9e8540da3c04f9eb90b5ad80782a/family2/217312c2aebc436ab7d37237cf5bdefa 2023-02-11 03:55:57,258 INFO [Listener at localhost/46205] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 217312c2aebc436ab7d37237cf5bdefa 2023-02-11 03:55:57,259 INFO [Listener at localhost/46205] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 08cc9e8540da3c04f9eb90b5ad80782a/family2 of 08cc9e8540da3c04f9eb90b5ad80782a into 217312c2aebc436ab7d37237cf5bdefa(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-02-11 03:55:57,259 DEBUG [Listener at localhost/46205] regionserver.HRegion(2288): Compaction status journal for 08cc9e8540da3c04f9eb90b5ad80782a: 2023-02-11 03:55:57,262 DEBUG [Listener at localhost/46205] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-02-11 03:55:57,265 DEBUG [Listener at localhost/46205] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-02-11 03:55:57,267 DEBUG [Listener at localhost/46205] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-02-11 03:55:57,268 DEBUG [Listener at localhost/46205] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-02-11 03:55:57,272 INFO [Listener at localhost/46205] compactions.PartitionedMobCompactor(142): is allFiles: false 2023-02-11 03:55:57,285 INFO [Listener at localhost/46205] compactions.PartitionedMobCompactor(306): The compaction type is PART_FILES, the request has 3 del files, 3 selected files, and 0 irrelevant files table 'ns:testMinorCompaction' and column 'family1' 2023-02-11 03:55:57,288 INFO [Listener at localhost/46205] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e202302111760d094e96349f196c9c6df7e3abb5c_del 2023-02-11 03:55:57,291 INFO [Listener at localhost/46205] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b20230211988c0389c6b44d1291d66a7b4acee814_del 2023-02-11 03:55:57,294 INFO [Listener at localhost/46205] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c202302111c6ff7965ebf424da0357ae03949d326_del 2023-02-11 03:55:57,294 INFO [Listener at localhost/46205] compactions.PartitionedMobCompactor(350): After merging, there are 3 del files. table='ns:testMinorCompaction' column='family1' 2023-02-11 03:55:57,294 DEBUG [Listener at localhost/46205] zookeeper.ReadOnlyZKClient(139): Connect 0x5282e94f to 127.0.0.1:53383 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-11 03:55:57,300 DEBUG [Listener at localhost/46205] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@5ff8af27, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-11 03:55:57,301 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c4ca4238a0b923820dcc509a6f75849b20230211 for table='ns:testMinorCompaction' and column='family1' 2023-02-11 03:55:57,301 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c81e728d9d4c2f636f067f89cc14862c20230211 for table='ns:testMinorCompaction' and column='family1' 2023-02-11 03:55:57,301 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20230211 for table='ns:testMinorCompaction' and column='family1' 2023-02-11 03:55:57,349 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/08cc9e8540da3c04f9eb90b5ad80782a/family2/40cb6bf4cd204ced9ed8f8b62e7eb74b, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/08cc9e8540da3c04f9eb90b5ad80782a/family2/2ae65cf0e4db479c8f62d84e1541cfe0, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/08cc9e8540da3c04f9eb90b5ad80782a/family2/be792f70fa374c46a3ecacbbae329638, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/08cc9e8540da3c04f9eb90b5ad80782a/family2/fa32da4a61cf4a7794826f9b7f890709, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/08cc9e8540da3c04f9eb90b5ad80782a/family2/a2ea90f611824324863856f104dfa5c8] to archive 2023-02-11 03:55:57,350 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-11 03:55:57,352 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/08cc9e8540da3c04f9eb90b5ad80782a/family2/40cb6bf4cd204ced9ed8f8b62e7eb74b to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/ns/testMinorCompaction/08cc9e8540da3c04f9eb90b5ad80782a/family2/40cb6bf4cd204ced9ed8f8b62e7eb74b 2023-02-11 03:55:57,353 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/08cc9e8540da3c04f9eb90b5ad80782a/family2/2ae65cf0e4db479c8f62d84e1541cfe0 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/ns/testMinorCompaction/08cc9e8540da3c04f9eb90b5ad80782a/family2/2ae65cf0e4db479c8f62d84e1541cfe0 2023-02-11 03:55:57,354 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/08cc9e8540da3c04f9eb90b5ad80782a/family2/be792f70fa374c46a3ecacbbae329638 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/ns/testMinorCompaction/08cc9e8540da3c04f9eb90b5ad80782a/family2/be792f70fa374c46a3ecacbbae329638 2023-02-11 03:55:57,356 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/08cc9e8540da3c04f9eb90b5ad80782a/family2/fa32da4a61cf4a7794826f9b7f890709 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/ns/testMinorCompaction/08cc9e8540da3c04f9eb90b5ad80782a/family2/fa32da4a61cf4a7794826f9b7f890709 2023-02-11 03:55:57,357 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/08cc9e8540da3c04f9eb90b5ad80782a/family2/a2ea90f611824324863856f104dfa5c8 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/ns/testMinorCompaction/08cc9e8540da3c04f9eb90b5ad80782a/family2/a2ea90f611824324863856f104dfa5c8 2023-02-11 03:55:57,751 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-02-11 03:55:57,751 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x18639285 to 127.0.0.1:53383 2023-02-11 03:55:57,752 DEBUG [Finalizer] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-11 03:55:57,752 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-02-11 03:55:57,752 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x32f4cb3b to 127.0.0.1:53383 2023-02-11 03:55:57,752 DEBUG [Finalizer] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-11 03:55:57,754 INFO [MobFileCompactionChore-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230211a6b7ba568eb047dc85a6d6859c6508f5 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/d41d8cd98f00b204e9800998ecf8427e20230211a6b7ba568eb047dc85a6d6859c6508f5 2023-02-11 03:55:57,755 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20230211 table='ns:testMinorCompaction' column='family1' 2023-02-11 03:55:57,757 INFO [MobFileCompactionChore-pool-2] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202302110bd18a9e52364a19adefede45db96283 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c4ca4238a0b923820dcc509a6f75849b202302110bd18a9e52364a19adefede45db96283 2023-02-11 03:55:57,758 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c4ca4238a0b923820dcc509a6f75849b20230211 table='ns:testMinorCompaction' column='family1' 2023-02-11 03:55:57,758 INFO [MobFileCompactionChore-pool-1] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230211e843c19843a74df4814379dabb9bcd0c to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c81e728d9d4c2f636f067f89cc14862c20230211e843c19843a74df4814379dabb9bcd0c 2023-02-11 03:55:57,759 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20230211 table='ns:testMinorCompaction' column='family1' 2023-02-11 03:55:57,770 DEBUG [MobFileCompactionChore-pool-1] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-11 03:55:57,773 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:45652, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-11 03:55:57,779 DEBUG [MobFileCompactionChore-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-02-11 03:55:57,787 INFO [RS-EventLoopGroup-1-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:52940, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-02-11 03:55:57,805 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c81e728d9d4c2f636f067f89cc14862c20230211/family1/f6437e5d7466476ba5fc824ce71774b6 first=Optional[2\x00\x00\x00\x01] last=Optional[2\x00\x00\x00\x01] 2023-02-11 03:55:57,813 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=ns:testMinorCompaction,2,1676087739490.08cc9e8540da3c04f9eb90b5ad80782a., hostname=jenkins-hbase4.apache.org,37011,1676087618428, seqNum=2 for row 2 with hfile group [{family1,hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c81e728d9d4c2f636f067f89cc14862c20230211/family1/f6437e5d7466476ba5fc824ce71774b6}] 2023-02-11 03:55:57,820 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c4ca4238a0b923820dcc509a6f75849b20230211/family1/c2b3fdef9e9c444fa599245f9e1421df first=Optional[1\x00\x00\x00\x01] last=Optional[1\x00\x00\x00\x01] 2023-02-11 03:55:57,825 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/.bulkload/ns/testMinorCompaction/d41d8cd98f00b204e9800998ecf8427e20230211/family1/6a16403e5df5409db2204e6b40ff0661 first=Optional[0\x00\x00\x00\x01] last=Optional[0\x00\x00\x00\x01] 2023-02-11 03:55:57,825 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=ns:testMinorCompaction,1,1676087739490.3fa91eb317d63eaff7b1021985f3ad00., hostname=jenkins-hbase4.apache.org,37011,1676087618428, seqNum=2 for row 1 with hfile group [{family1,hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c4ca4238a0b923820dcc509a6f75849b20230211/family1/c2b3fdef9e9c444fa599245f9e1421df}] 2023-02-11 03:55:57,831 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=ns:testMinorCompaction,,1676087739490.6938f3b95bbc15aff8fb4a9d24d9aaf5., hostname=jenkins-hbase4.apache.org,37011,1676087618428, seqNum=2 for row with hfile group [{family1,hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/.bulkload/ns/testMinorCompaction/d41d8cd98f00b204e9800998ecf8427e20230211/family1/6a16403e5df5409db2204e6b40ff0661}] 2023-02-11 03:55:57,833 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37011] regionserver.HStore(763): Validating hfile at hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c81e728d9d4c2f636f067f89cc14862c20230211/family1/f6437e5d7466476ba5fc824ce71774b6 for inclusion in 08cc9e8540da3c04f9eb90b5ad80782a/family1 2023-02-11 03:55:57,833 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.HStore(763): Validating hfile at hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c4ca4238a0b923820dcc509a6f75849b20230211/family1/c2b3fdef9e9c444fa599245f9e1421df for inclusion in 3fa91eb317d63eaff7b1021985f3ad00/family1 2023-02-11 03:55:57,834 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37011] regionserver.HStore(763): Validating hfile at hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/.bulkload/ns/testMinorCompaction/d41d8cd98f00b204e9800998ecf8427e20230211/family1/6a16403e5df5409db2204e6b40ff0661 for inclusion in 6938f3b95bbc15aff8fb4a9d24d9aaf5/family1 2023-02-11 03:55:57,836 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37011] regionserver.HStore(775): HFile bounds: first=2\x00\x00\x00\x01 last=2\x00\x00\x00\x01 2023-02-11 03:55:57,836 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.HStore(775): HFile bounds: first=1\x00\x00\x00\x01 last=1\x00\x00\x00\x01 2023-02-11 03:55:57,836 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.HStore(777): Region bounds: first=1 last=2 2023-02-11 03:55:57,836 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37011] regionserver.HStore(777): Region bounds: first=2 last= 2023-02-11 03:55:57,836 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.HRegion(2445): Flush status journal for 3fa91eb317d63eaff7b1021985f3ad00: 2023-02-11 03:55:57,836 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37011] regionserver.HRegion(2445): Flush status journal for 08cc9e8540da3c04f9eb90b5ad80782a: 2023-02-11 03:55:57,837 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37011] regionserver.HStore(775): HFile bounds: first=0\x00\x00\x00\x01 last=0\x00\x00\x00\x01 2023-02-11 03:55:57,837 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37011] regionserver.HStore(777): Region bounds: first= last=1 2023-02-11 03:55:57,837 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37011] regionserver.HRegion(2445): Flush status journal for 6938f3b95bbc15aff8fb4a9d24d9aaf5: 2023-02-11 03:55:57,843 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c4ca4238a0b923820dcc509a6f75849b20230211/family1/c2b3fdef9e9c444fa599245f9e1421df to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/staging/jenkins__ns_testMinorCompaction__qmgqo1nmho95mqi8r423j8b64833ep46hddfsh4is10uf0q02oh2q4v3ks6u038k/family1/c2b3fdef9e9c444fa599245f9e1421df 2023-02-11 03:55:57,843 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37011] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c81e728d9d4c2f636f067f89cc14862c20230211/family1/f6437e5d7466476ba5fc824ce71774b6 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/staging/jenkins__ns_testMinorCompaction__glp22t0a8s54vma7igng1k5krin49a0h5d9c5h4829a3hoqtu6nm4fdnfclqashf/family1/f6437e5d7466476ba5fc824ce71774b6 2023-02-11 03:55:57,843 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37011] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/.bulkload/ns/testMinorCompaction/d41d8cd98f00b204e9800998ecf8427e20230211/family1/6a16403e5df5409db2204e6b40ff0661 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/staging/jenkins__ns_testMinorCompaction__bvbac4q8f6egng130i0109k9c8d314qgs9kronglngaavbgfmprgob5ssuc1592m/family1/6a16403e5df5409db2204e6b40ff0661 2023-02-11 03:55:57,845 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/staging/jenkins__ns_testMinorCompaction__qmgqo1nmho95mqi8r423j8b64833ep46hddfsh4is10uf0q02oh2q4v3ks6u038k/family1/c2b3fdef9e9c444fa599245f9e1421df as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/3fa91eb317d63eaff7b1021985f3ad00/family1/9391d54f463e499f9e3ed8941ae93dc2_SeqId_28_ 2023-02-11 03:55:57,845 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37011] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/staging/jenkins__ns_testMinorCompaction__glp22t0a8s54vma7igng1k5krin49a0h5d9c5h4829a3hoqtu6nm4fdnfclqashf/family1/f6437e5d7466476ba5fc824ce71774b6 as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/08cc9e8540da3c04f9eb90b5ad80782a/family1/0939ef04616043fd979fbb41b47eb716_SeqId_28_ 2023-02-11 03:55:57,845 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37011] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/staging/jenkins__ns_testMinorCompaction__bvbac4q8f6egng130i0109k9c8d314qgs9kronglngaavbgfmprgob5ssuc1592m/family1/6a16403e5df5409db2204e6b40ff0661 as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/6938f3b95bbc15aff8fb4a9d24d9aaf5/family1/4d3c805a1ce44f88a5ecc5a9dac1b822_SeqId_28_ 2023-02-11 03:55:57,845 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.HStore(849): Loaded HFile hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/staging/jenkins__ns_testMinorCompaction__qmgqo1nmho95mqi8r423j8b64833ep46hddfsh4is10uf0q02oh2q4v3ks6u038k/family1/c2b3fdef9e9c444fa599245f9e1421df into 3fa91eb317d63eaff7b1021985f3ad00/family1 as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/3fa91eb317d63eaff7b1021985f3ad00/family1/9391d54f463e499f9e3ed8941ae93dc2_SeqId_28_ - updating store file list. 2023-02-11 03:55:57,846 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37011] regionserver.HStore(849): Loaded HFile hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/staging/jenkins__ns_testMinorCompaction__glp22t0a8s54vma7igng1k5krin49a0h5d9c5h4829a3hoqtu6nm4fdnfclqashf/family1/f6437e5d7466476ba5fc824ce71774b6 into 08cc9e8540da3c04f9eb90b5ad80782a/family1 as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/08cc9e8540da3c04f9eb90b5ad80782a/family1/0939ef04616043fd979fbb41b47eb716_SeqId_28_ - updating store file list. 2023-02-11 03:55:57,846 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37011] regionserver.HStore(849): Loaded HFile hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/staging/jenkins__ns_testMinorCompaction__bvbac4q8f6egng130i0109k9c8d314qgs9kronglngaavbgfmprgob5ssuc1592m/family1/6a16403e5df5409db2204e6b40ff0661 into 6938f3b95bbc15aff8fb4a9d24d9aaf5/family1 as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/6938f3b95bbc15aff8fb4a9d24d9aaf5/family1/4d3c805a1ce44f88a5ecc5a9dac1b822_SeqId_28_ - updating store file list. 2023-02-11 03:55:57,849 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37011] regionserver.HStore(882): Loaded HFile hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/08cc9e8540da3c04f9eb90b5ad80782a/family1/0939ef04616043fd979fbb41b47eb716_SeqId_28_ into 08cc9e8540da3c04f9eb90b5ad80782a/family1 2023-02-11 03:55:57,849 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37011] regionserver.HStore(855): Successfully loaded hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/staging/jenkins__ns_testMinorCompaction__glp22t0a8s54vma7igng1k5krin49a0h5d9c5h4829a3hoqtu6nm4fdnfclqashf/family1/f6437e5d7466476ba5fc824ce71774b6 into 08cc9e8540da3c04f9eb90b5ad80782a/family1 (new location: hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/08cc9e8540da3c04f9eb90b5ad80782a/family1/0939ef04616043fd979fbb41b47eb716_SeqId_28_) 2023-02-11 03:55:57,849 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.HStore(882): Loaded HFile hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/3fa91eb317d63eaff7b1021985f3ad00/family1/9391d54f463e499f9e3ed8941ae93dc2_SeqId_28_ into 3fa91eb317d63eaff7b1021985f3ad00/family1 2023-02-11 03:55:57,849 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.HStore(855): Successfully loaded hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/staging/jenkins__ns_testMinorCompaction__qmgqo1nmho95mqi8r423j8b64833ep46hddfsh4is10uf0q02oh2q4v3ks6u038k/family1/c2b3fdef9e9c444fa599245f9e1421df into 3fa91eb317d63eaff7b1021985f3ad00/family1 (new location: hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/3fa91eb317d63eaff7b1021985f3ad00/family1/9391d54f463e499f9e3ed8941ae93dc2_SeqId_28_) 2023-02-11 03:55:57,849 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37011] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/staging/jenkins__ns_testMinorCompaction__glp22t0a8s54vma7igng1k5krin49a0h5d9c5h4829a3hoqtu6nm4fdnfclqashf/family1/f6437e5d7466476ba5fc824ce71774b6 2023-02-11 03:55:57,849 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37011] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/staging/jenkins__ns_testMinorCompaction__qmgqo1nmho95mqi8r423j8b64833ep46hddfsh4is10uf0q02oh2q4v3ks6u038k/family1/c2b3fdef9e9c444fa599245f9e1421df 2023-02-11 03:55:57,850 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37011] regionserver.HStore(882): Loaded HFile hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/6938f3b95bbc15aff8fb4a9d24d9aaf5/family1/4d3c805a1ce44f88a5ecc5a9dac1b822_SeqId_28_ into 6938f3b95bbc15aff8fb4a9d24d9aaf5/family1 2023-02-11 03:55:57,850 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37011] regionserver.HStore(855): Successfully loaded hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/staging/jenkins__ns_testMinorCompaction__bvbac4q8f6egng130i0109k9c8d314qgs9kronglngaavbgfmprgob5ssuc1592m/family1/6a16403e5df5409db2204e6b40ff0661 into 6938f3b95bbc15aff8fb4a9d24d9aaf5/family1 (new location: hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/6938f3b95bbc15aff8fb4a9d24d9aaf5/family1/4d3c805a1ce44f88a5ecc5a9dac1b822_SeqId_28_) 2023-02-11 03:55:57,850 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37011] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/staging/jenkins__ns_testMinorCompaction__bvbac4q8f6egng130i0109k9c8d314qgs9kronglngaavbgfmprgob5ssuc1592m/family1/6a16403e5df5409db2204e6b40ff0661 2023-02-11 03:55:57,851 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c4ca4238a0b923820dcc509a6f75849b20230211 table='ns:testMinorCompaction' column='family1' 2023-02-11 03:55:57,851 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(360): Archiving compacted files. 2023-02-11 03:55:57,852 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20230211 table='ns:testMinorCompaction' column='family1' 2023-02-11 03:55:57,852 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-11 03:55:57,853 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c4ca4238a0b923820dcc509a6f75849b202302118b2f27fc9704450fbba349a924ac5f5d to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c4ca4238a0b923820dcc509a6f75849b202302118b2f27fc9704450fbba349a924ac5f5d 2023-02-11 03:55:57,853 INFO [MobFileCompactionChore-pool-2] mob.MobUtils(508): Table ns:testMinorCompaction 1 expired mob files are deleted 2023-02-11 03:55:57,853 DEBUG [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c4ca4238a0b923820dcc509a6f75849b20230211' 2023-02-11 03:55:57,853 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/d41d8cd98f00b204e9800998ecf8427e20230211742c6d88e07649a28098ce0ba5e992e5 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/d41d8cd98f00b204e9800998ecf8427e20230211742c6d88e07649a28098ce0ba5e992e5 2023-02-11 03:55:57,853 INFO [MobFileCompactionChore-pool-0] mob.MobUtils(508): Table ns:testMinorCompaction 1 expired mob files are deleted 2023-02-11 03:55:57,853 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=c4ca4238a0b923820dcc509a6f75849b20230211 for table='ns:testMinorCompaction' and column='family1' 2023-02-11 03:55:57,854 DEBUG [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/.bulkload/ns/testMinorCompaction/d41d8cd98f00b204e9800998ecf8427e20230211' 2023-02-11 03:55:57,854 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20230211 table='ns:testMinorCompaction' column='family1' 2023-02-11 03:55:57,854 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20230211 for table='ns:testMinorCompaction' and column='family1' 2023-02-11 03:55:57,854 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-11 03:55:57,855 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c81e728d9d4c2f636f067f89cc14862c2023021170c6e59bf6274b85aecdb9321c195f05 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c81e728d9d4c2f636f067f89cc14862c2023021170c6e59bf6274b85aecdb9321c195f05 2023-02-11 03:55:57,855 INFO [MobFileCompactionChore-pool-1] mob.MobUtils(508): Table ns:testMinorCompaction 1 expired mob files are deleted 2023-02-11 03:55:57,855 DEBUG [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c81e728d9d4c2f636f067f89cc14862c20230211' 2023-02-11 03:55:57,856 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=c81e728d9d4c2f636f067f89cc14862c20230211 for table='ns:testMinorCompaction' and column='family1' 2023-02-11 03:55:57,856 INFO [Listener at localhost/46205] compactions.PartitionedMobCompactor(354): After compaction, there are 3 mob files. table='ns:testMinorCompaction' column='family1' 2023-02-11 03:55:57,858 DEBUG [Listener at localhost/46205] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-02-11 03:55:57,860 DEBUG [Listener at localhost/46205] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-02-11 03:55:57,863 DEBUG [Listener at localhost/46205] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-02-11 03:55:57,863 DEBUG [Listener at localhost/46205] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-02-11 03:55:57,881 INFO [Listener at localhost/46205] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMinorCompaction Thread=251 (was 285), OpenFileDescriptor=626 (was 766), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=109 (was 111), ProcessCount=168 (was 168), AvailableMemoryMB=2865 (was 2987) 2023-02-11 03:55:57,895 INFO [Listener at localhost/46205] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMinorCompactionWithMonthlyPolicy Thread=251, OpenFileDescriptor=626, MaxFileDescriptor=60000, SystemLoadAverage=109, ProcessCount=168, AvailableMemoryMB=2865 2023-02-11 03:55:57,897 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39921] master.HMaster$4(2102): Client=jenkins//172.31.14.131 create 'testMinorCompactionWithMonthlyPolicy', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'} 2023-02-11 03:55:57,897 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39921] procedure2.ProcedureExecutor(1029): Stored pid=112, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testMinorCompactionWithMonthlyPolicy 2023-02-11 03:55:57,899 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=112, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testMinorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_PRE_OPERATION 2023-02-11 03:55:57,899 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39921] master.MasterRpcServices(697): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testMinorCompactionWithMonthlyPolicy" procId is: 112 2023-02-11 03:55:57,899 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=112, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testMinorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-11 03:55:57,900 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39921] master.MasterRpcServices(1227): Checking to see if procedure is done pid=112 2023-02-11 03:55:57,900 DEBUG [HFileArchiver-19] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/.tmp/data/default/testMinorCompactionWithMonthlyPolicy/9c79e0360165a5fa04cbe6dbc395fafc 2023-02-11 03:55:57,901 DEBUG [HFileArchiver-19] backup.HFileArchiver(153): Directory hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/.tmp/data/default/testMinorCompactionWithMonthlyPolicy/9c79e0360165a5fa04cbe6dbc395fafc empty. 2023-02-11 03:55:57,901 DEBUG [HFileArchiver-19] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/.tmp/data/default/testMinorCompactionWithMonthlyPolicy/9c79e0360165a5fa04cbe6dbc395fafc 2023-02-11 03:55:57,901 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testMinorCompactionWithMonthlyPolicy regions 2023-02-11 03:55:58,001 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39921] master.MasterRpcServices(1227): Checking to see if procedure is done pid=112 2023-02-11 03:55:58,202 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39921] master.MasterRpcServices(1227): Checking to see if procedure is done pid=112 2023-02-11 03:55:58,307 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/.tmp/data/default/testMinorCompactionWithMonthlyPolicy/.tabledesc/.tableinfo.0000000001 2023-02-11 03:55:58,309 INFO [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(7671): creating {ENCODED => 9c79e0360165a5fa04cbe6dbc395fafc, NAME => 'testMinorCompactionWithMonthlyPolicy,,1676087757896.9c79e0360165a5fa04cbe6dbc395fafc.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testMinorCompactionWithMonthlyPolicy', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'}, regionDir=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/.tmp 2023-02-11 03:55:58,503 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39921] master.MasterRpcServices(1227): Checking to see if procedure is done pid=112 2023-02-11 03:55:58,715 DEBUG [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(865): Instantiated testMinorCompactionWithMonthlyPolicy,,1676087757896.9c79e0360165a5fa04cbe6dbc395fafc.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-11 03:55:58,715 DEBUG [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1603): Closing 9c79e0360165a5fa04cbe6dbc395fafc, disabling compactions & flushes 2023-02-11 03:55:58,715 INFO [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1625): Closing region testMinorCompactionWithMonthlyPolicy,,1676087757896.9c79e0360165a5fa04cbe6dbc395fafc. 2023-02-11 03:55:58,715 DEBUG [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testMinorCompactionWithMonthlyPolicy,,1676087757896.9c79e0360165a5fa04cbe6dbc395fafc. 2023-02-11 03:55:58,715 DEBUG [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1713): Acquired close lock on testMinorCompactionWithMonthlyPolicy,,1676087757896.9c79e0360165a5fa04cbe6dbc395fafc. after waiting 0 ms 2023-02-11 03:55:58,715 DEBUG [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1723): Updates disabled for region testMinorCompactionWithMonthlyPolicy,,1676087757896.9c79e0360165a5fa04cbe6dbc395fafc. 2023-02-11 03:55:58,715 INFO [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1837): Closed testMinorCompactionWithMonthlyPolicy,,1676087757896.9c79e0360165a5fa04cbe6dbc395fafc. 2023-02-11 03:55:58,715 DEBUG [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1557): Region close journal for 9c79e0360165a5fa04cbe6dbc395fafc: Waiting for close lock at 1676087758715Disabling compacts and flushes for region at 1676087758715Disabling writes for close at 1676087758715Writing region close event to WAL at 1676087758715Closed at 1676087758715 2023-02-11 03:55:58,717 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=112, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testMinorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_ADD_TO_META 2023-02-11 03:55:58,718 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMinorCompactionWithMonthlyPolicy,,1676087757896.9c79e0360165a5fa04cbe6dbc395fafc.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1676087758718"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676087758718"}]},"ts":"1676087758718"} 2023-02-11 03:55:58,720 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-11 03:55:58,720 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=112, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testMinorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-11 03:55:58,720 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMinorCompactionWithMonthlyPolicy","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676087758720"}]},"ts":"1676087758720"} 2023-02-11 03:55:58,721 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testMinorCompactionWithMonthlyPolicy, state=ENABLING in hbase:meta 2023-02-11 03:55:58,724 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=113, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMinorCompactionWithMonthlyPolicy, region=9c79e0360165a5fa04cbe6dbc395fafc, ASSIGN}] 2023-02-11 03:55:58,726 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=113, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMinorCompactionWithMonthlyPolicy, region=9c79e0360165a5fa04cbe6dbc395fafc, ASSIGN 2023-02-11 03:55:58,726 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=113, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMinorCompactionWithMonthlyPolicy, region=9c79e0360165a5fa04cbe6dbc395fafc, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,37011,1676087618428; forceNewPlan=false, retain=false 2023-02-11 03:55:58,877 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=113 updating hbase:meta row=9c79e0360165a5fa04cbe6dbc395fafc, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:55:58,878 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMinorCompactionWithMonthlyPolicy,,1676087757896.9c79e0360165a5fa04cbe6dbc395fafc.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1676087758877"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676087758877"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676087758877"}]},"ts":"1676087758877"} 2023-02-11 03:55:58,880 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=114, ppid=113, state=RUNNABLE; OpenRegionProcedure 9c79e0360165a5fa04cbe6dbc395fafc, server=jenkins-hbase4.apache.org,37011,1676087618428}] 2023-02-11 03:55:59,004 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39921] master.MasterRpcServices(1227): Checking to see if procedure is done pid=112 2023-02-11 03:55:59,036 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMinorCompactionWithMonthlyPolicy,,1676087757896.9c79e0360165a5fa04cbe6dbc395fafc. 2023-02-11 03:55:59,036 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 9c79e0360165a5fa04cbe6dbc395fafc, NAME => 'testMinorCompactionWithMonthlyPolicy,,1676087757896.9c79e0360165a5fa04cbe6dbc395fafc.', STARTKEY => '', ENDKEY => ''} 2023-02-11 03:55:59,037 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMinorCompactionWithMonthlyPolicy 9c79e0360165a5fa04cbe6dbc395fafc 2023-02-11 03:55:59,037 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testMinorCompactionWithMonthlyPolicy,,1676087757896.9c79e0360165a5fa04cbe6dbc395fafc.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-11 03:55:59,037 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 9c79e0360165a5fa04cbe6dbc395fafc 2023-02-11 03:55:59,037 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 9c79e0360165a5fa04cbe6dbc395fafc 2023-02-11 03:55:59,038 INFO [StoreOpener-9c79e0360165a5fa04cbe6dbc395fafc-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 9c79e0360165a5fa04cbe6dbc395fafc 2023-02-11 03:55:59,040 DEBUG [StoreOpener-9c79e0360165a5fa04cbe6dbc395fafc-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMinorCompactionWithMonthlyPolicy/9c79e0360165a5fa04cbe6dbc395fafc/family1 2023-02-11 03:55:59,040 DEBUG [StoreOpener-9c79e0360165a5fa04cbe6dbc395fafc-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMinorCompactionWithMonthlyPolicy/9c79e0360165a5fa04cbe6dbc395fafc/family1 2023-02-11 03:55:59,040 INFO [StoreOpener-9c79e0360165a5fa04cbe6dbc395fafc-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 9c79e0360165a5fa04cbe6dbc395fafc columnFamilyName family1 2023-02-11 03:55:59,042 INFO [StoreOpener-9c79e0360165a5fa04cbe6dbc395fafc-1] regionserver.HStore(310): Store=9c79e0360165a5fa04cbe6dbc395fafc/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 03:55:59,043 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMinorCompactionWithMonthlyPolicy/9c79e0360165a5fa04cbe6dbc395fafc 2023-02-11 03:55:59,043 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMinorCompactionWithMonthlyPolicy/9c79e0360165a5fa04cbe6dbc395fafc 2023-02-11 03:55:59,045 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 9c79e0360165a5fa04cbe6dbc395fafc 2023-02-11 03:55:59,046 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMinorCompactionWithMonthlyPolicy/9c79e0360165a5fa04cbe6dbc395fafc/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-11 03:55:59,046 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 9c79e0360165a5fa04cbe6dbc395fafc; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=72110396, jitterRate=0.07452863454818726}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-11 03:55:59,046 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 9c79e0360165a5fa04cbe6dbc395fafc: Running coprocessor pre-open hook at 1676087759037Writing region info on filesystem at 1676087759037Initializing all the Stores at 1676087759038 (+1 ms)Instantiating store for column family {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'} at 1676087759038Cleaning up temporary data from old regions at 1676087759043 (+5 ms)Cleaning up detritus from prior splits at 1676087759043Running coprocessor post-open hooks at 1676087759046 (+3 ms)Region opened successfully at 1676087759046 2023-02-11 03:55:59,047 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMinorCompactionWithMonthlyPolicy,,1676087757896.9c79e0360165a5fa04cbe6dbc395fafc., pid=114, masterSystemTime=1676087759032 2023-02-11 03:55:59,048 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMinorCompactionWithMonthlyPolicy,,1676087757896.9c79e0360165a5fa04cbe6dbc395fafc. 2023-02-11 03:55:59,049 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMinorCompactionWithMonthlyPolicy,,1676087757896.9c79e0360165a5fa04cbe6dbc395fafc. 2023-02-11 03:55:59,049 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=113 updating hbase:meta row=9c79e0360165a5fa04cbe6dbc395fafc, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:55:59,049 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMinorCompactionWithMonthlyPolicy,,1676087757896.9c79e0360165a5fa04cbe6dbc395fafc.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1676087759049"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676087759049"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676087759049"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676087759049"}]},"ts":"1676087759049"} 2023-02-11 03:55:59,052 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=114, resume processing ppid=113 2023-02-11 03:55:59,052 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=114, ppid=113, state=SUCCESS; OpenRegionProcedure 9c79e0360165a5fa04cbe6dbc395fafc, server=jenkins-hbase4.apache.org,37011,1676087618428 in 170 msec 2023-02-11 03:55:59,053 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=113, resume processing ppid=112 2023-02-11 03:55:59,053 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testMinorCompactionWithMonthlyPolicy, region=9c79e0360165a5fa04cbe6dbc395fafc, ASSIGN in 328 msec 2023-02-11 03:55:59,054 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=112, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testMinorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-11 03:55:59,054 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMinorCompactionWithMonthlyPolicy","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676087759054"}]},"ts":"1676087759054"} 2023-02-11 03:55:59,055 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=testMinorCompactionWithMonthlyPolicy, state=ENABLED in hbase:meta 2023-02-11 03:55:59,061 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=112, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testMinorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_POST_OPERATION 2023-02-11 03:55:59,062 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=112, state=SUCCESS; CreateTableProcedure table=testMinorCompactionWithMonthlyPolicy in 1.1640 sec 2023-02-11 03:56:00,005 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39921] master.MasterRpcServices(1227): Checking to see if procedure is done pid=112 2023-02-11 03:56:00,006 INFO [Listener at localhost/46205] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testMinorCompactionWithMonthlyPolicy, procId: 112 completed 2023-02-11 03:56:00,031 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39921] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 03:56:00,038 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39921] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:00,039 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-02-11 03:56:00,039 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:56:00,039 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-02-11 03:56:00,039 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-02-11 03:56:00,039 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:00,039 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:00,040 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:56:00,040 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:00,041 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:56:00,041 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:56:00,041 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:00,041 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 03:56:00,041 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:00,041 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:00,042 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 03:56:00,042 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:00,042 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:00,042 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:00,042 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-02-11 03:56:00,042 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:56:00,042 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-02-11 03:56:00,042 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 03:56:00,042 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 03:56:00,042 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1676087757896.9c79e0360165a5fa04cbe6dbc395fafc. 2023-02-11 03:56:00,043 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1676087757896.9c79e0360165a5fa04cbe6dbc395fafc. started... 2023-02-11 03:56:00,043 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 9c79e0360165a5fa04cbe6dbc395fafc 1/1 column families, dataSize=74.11 KB heapSize=148.69 KB 2023-02-11 03:56:00,076 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151130ff996190d2dd49eba541c38c5f7e8cc5 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151130ff996190d2dd49eba541c38c5f7e8cc5 2023-02-11 03:56:00,481 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=74.1 K, hasBloomFilter=true, into tmp file hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMinorCompactionWithMonthlyPolicy/9c79e0360165a5fa04cbe6dbc395fafc/.tmp/family1/436ff91765344ba6be250001c5672c69 2023-02-11 03:56:00,486 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMinorCompactionWithMonthlyPolicy/9c79e0360165a5fa04cbe6dbc395fafc/.tmp/family1/436ff91765344ba6be250001c5672c69 as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMinorCompactionWithMonthlyPolicy/9c79e0360165a5fa04cbe6dbc395fafc/family1/436ff91765344ba6be250001c5672c69 2023-02-11 03:56:00,489 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMinorCompactionWithMonthlyPolicy/9c79e0360165a5fa04cbe6dbc395fafc/family1/436ff91765344ba6be250001c5672c69, entries=1000, sequenceid=5, filesize=166.0 K 2023-02-11 03:56:00,490 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~74.11 KB/75890, heapSize ~148.67 KB/152240, currentSize=0 B/0 for 9c79e0360165a5fa04cbe6dbc395fafc in 447ms, sequenceid=5, compaction requested=false 2023-02-11 03:56:00,490 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testMinorCompactionWithMonthlyPolicy' 2023-02-11 03:56:00,491 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 9c79e0360165a5fa04cbe6dbc395fafc: 2023-02-11 03:56:00,491 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1676087757896.9c79e0360165a5fa04cbe6dbc395fafc. 2023-02-11 03:56:00,491 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 03:56:00,491 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 03:56:00,491 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:00,491 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-02-11 03:56:00,491 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37011,1676087618428' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-02-11 03:56:00,493 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:00,493 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:00,493 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:00,493 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:56:00,493 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:56:00,493 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:00,493 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 03:56:00,493 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:56:00,493 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:56:00,494 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:00,494 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:00,494 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:56:00,494 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37011,1676087618428' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-02-11 03:56:00,494 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2478e61[Count = 0] remaining members to acquire global barrier 2023-02-11 03:56:00,494 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-02-11 03:56:00,494 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:00,495 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:00,495 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:00,495 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:00,496 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-02-11 03:56:00,496 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-02-11 03:56:00,496 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,37011,1676087618428' in zk 2023-02-11 03:56:00,496 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:00,496 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 03:56:00,497 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-02-11 03:56:00,497 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:00,497 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:56:00,498 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:00,498 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:56:00,498 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:56:00,498 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-02-11 03:56:00,498 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:56:00,498 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:56:00,498 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:00,499 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:00,499 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:56:00,499 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:00,500 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:00,500 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,37011,1676087618428': 2023-02-11 03:56:00,500 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37011,1676087618428' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-02-11 03:56:00,500 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-02-11 03:56:00,500 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 03:56:00,500 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 03:56:00,500 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:00,500 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 03:56:00,503 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:00,503 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:00,503 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:56:00,503 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:00,503 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:00,503 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:00,503 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:56:00,503 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:56:00,503 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:00,503 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:56:00,503 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:56:00,503 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:56:00,504 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:00,504 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:00,504 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:56:00,504 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:00,504 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:00,505 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:00,505 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:56:00,505 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:00,505 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:00,506 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:00,506 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:00,507 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:00,507 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:00,507 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:00,507 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:00,507 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:00,507 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:56:00,507 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:56:00,507 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-02-11 03:56:00,507 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39921] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-02-11 03:56:00,507 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39921] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 03:56:00,507 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:56:00,507 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:56:00,507 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:56:00,507 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-02-11 03:56:00,507 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 03:56:00,507 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:56:00,507 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:56:00,607 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 03:56:00,608 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39921] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 03:56:00,613 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39921] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 03:56:00,615 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39921] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-02-11 03:56:00,615 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39921] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:00,615 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-02-11 03:56:00,615 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:56:00,615 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-02-11 03:56:00,615 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-02-11 03:56:00,615 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:00,615 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:00,617 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:56:00,617 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:56:00,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:56:00,617 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:00,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:00,617 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:00,617 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 03:56:00,617 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:00,618 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 03:56:00,618 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:00,618 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:00,618 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-02-11 03:56:00,618 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:00,618 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-02-11 03:56:00,618 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:56:00,618 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-02-11 03:56:00,618 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 03:56:00,618 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 03:56:00,618 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1676087757896.9c79e0360165a5fa04cbe6dbc395fafc. 2023-02-11 03:56:00,618 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1676087757896.9c79e0360165a5fa04cbe6dbc395fafc. started... 2023-02-11 03:56:00,618 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 9c79e0360165a5fa04cbe6dbc395fafc 1/1 column families, dataSize=74 B heapSize=408 B 2023-02-11 03:56:01,036 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015112805f86d62a18e43d9ab11bf6d4c610fc5 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e2015112805f86d62a18e43d9ab11bf6d4c610fc5 2023-02-11 03:56:01,445 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=74, hasBloomFilter=true, into tmp file hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMinorCompactionWithMonthlyPolicy/9c79e0360165a5fa04cbe6dbc395fafc/.tmp/family1/50c1fe16b4ef47f18b7acf254d4a325a 2023-02-11 03:56:01,450 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMinorCompactionWithMonthlyPolicy/9c79e0360165a5fa04cbe6dbc395fafc/.tmp/family1/50c1fe16b4ef47f18b7acf254d4a325a as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMinorCompactionWithMonthlyPolicy/9c79e0360165a5fa04cbe6dbc395fafc/family1/50c1fe16b4ef47f18b7acf254d4a325a 2023-02-11 03:56:01,454 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMinorCompactionWithMonthlyPolicy/9c79e0360165a5fa04cbe6dbc395fafc/family1/50c1fe16b4ef47f18b7acf254d4a325a, entries=1, sequenceid=9, filesize=5.0 K 2023-02-11 03:56:01,455 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~74 B/74, heapSize ~392 B/392, currentSize=0 B/0 for 9c79e0360165a5fa04cbe6dbc395fafc in 837ms, sequenceid=9, compaction requested=false 2023-02-11 03:56:01,455 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 9c79e0360165a5fa04cbe6dbc395fafc: 2023-02-11 03:56:01,455 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1676087757896.9c79e0360165a5fa04cbe6dbc395fafc. 2023-02-11 03:56:01,455 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 03:56:01,455 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 03:56:01,456 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:01,456 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-02-11 03:56:01,456 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37011,1676087618428' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-02-11 03:56:01,457 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:01,457 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:01,457 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:01,457 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:56:01,457 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:56:01,457 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:01,458 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 03:56:01,458 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:56:01,458 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:56:01,458 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:01,459 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:01,459 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:56:01,459 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37011,1676087618428' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-02-11 03:56:01,459 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1cf2c1b8[Count = 0] remaining members to acquire global barrier 2023-02-11 03:56:01,459 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-02-11 03:56:01,459 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:01,460 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:01,460 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:01,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:01,461 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-02-11 03:56:01,461 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-02-11 03:56:01,461 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,37011,1676087618428' in zk 2023-02-11 03:56:01,461 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:01,461 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 03:56:01,462 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:01,462 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-02-11 03:56:01,462 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:56:01,462 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:01,462 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-02-11 03:56:01,462 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:56:01,462 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:56:01,462 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:56:01,463 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:56:01,463 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:01,463 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:01,464 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:56:01,464 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:01,464 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:01,465 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,37011,1676087618428': 2023-02-11 03:56:01,465 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37011,1676087618428' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-02-11 03:56:01,465 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-02-11 03:56:01,465 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 03:56:01,465 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 03:56:01,465 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:01,465 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 03:56:01,467 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:01,467 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:01,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:01,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:56:01,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:56:01,467 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:01,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:01,467 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:56:01,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:56:01,468 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:56:01,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:56:01,468 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:01,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:01,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:01,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:56:01,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:01,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:01,469 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:01,470 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:56:01,470 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:01,470 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:01,472 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:01,472 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:56:01,472 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:01,472 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:56:01,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:56:01,472 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:56:01,472 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-02-11 03:56:01,472 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:01,472 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39921] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-02-11 03:56:01,472 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39921] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 03:56:01,472 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:56:01,472 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:56:01,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:56:01,472 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:01,472 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:01,472 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:01,472 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:01,472 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-02-11 03:56:01,473 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 03:56:01,573 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 03:56:01,574 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39921] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 03:56:01,578 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39921] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 03:56:01,580 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39921] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-02-11 03:56:01,580 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39921] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:01,580 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-02-11 03:56:01,580 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:56:01,580 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-02-11 03:56:01,580 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-02-11 03:56:01,581 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:01,581 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:01,586 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:01,586 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:56:01,586 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:56:01,586 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:56:01,586 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:01,586 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 03:56:01,586 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:01,587 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:01,587 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 03:56:01,587 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:01,587 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:01,587 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-02-11 03:56:01,587 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:01,587 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-02-11 03:56:01,587 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:56:01,587 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-02-11 03:56:01,588 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 03:56:01,588 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 03:56:01,588 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1676087757896.9c79e0360165a5fa04cbe6dbc395fafc. 2023-02-11 03:56:01,588 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1676087757896.9c79e0360165a5fa04cbe6dbc395fafc. started... 2023-02-11 03:56:01,588 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 9c79e0360165a5fa04cbe6dbc395fafc 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-11 03:56:01,607 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015120152bb7fc8b7b24909a70ebe9c11054699 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e2015120152bb7fc8b7b24909a70ebe9c11054699 2023-02-11 03:56:02,012 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMinorCompactionWithMonthlyPolicy/9c79e0360165a5fa04cbe6dbc395fafc/.tmp/family1/c655cab8bd904a9cb5bad450397df19b 2023-02-11 03:56:02,017 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMinorCompactionWithMonthlyPolicy/9c79e0360165a5fa04cbe6dbc395fafc/.tmp/family1/c655cab8bd904a9cb5bad450397df19b as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMinorCompactionWithMonthlyPolicy/9c79e0360165a5fa04cbe6dbc395fafc/family1/c655cab8bd904a9cb5bad450397df19b 2023-02-11 03:56:02,022 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMinorCompactionWithMonthlyPolicy/9c79e0360165a5fa04cbe6dbc395fafc/family1/c655cab8bd904a9cb5bad450397df19b, entries=1, sequenceid=13, filesize=4.9 K 2023-02-11 03:56:02,023 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 9c79e0360165a5fa04cbe6dbc395fafc in 435ms, sequenceid=13, compaction requested=true 2023-02-11 03:56:02,023 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 9c79e0360165a5fa04cbe6dbc395fafc: 2023-02-11 03:56:02,023 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1676087757896.9c79e0360165a5fa04cbe6dbc395fafc. 2023-02-11 03:56:02,023 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 03:56:02,023 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 03:56:02,023 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:02,023 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-02-11 03:56:02,023 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37011,1676087618428' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-02-11 03:56:02,025 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:02,025 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:02,025 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:56:02,025 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:56:02,025 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:02,025 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:56:02,025 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:02,026 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 03:56:02,026 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:56:02,026 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:02,026 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:02,027 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:56:02,027 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37011,1676087618428' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-02-11 03:56:02,027 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5e4567eb[Count = 0] remaining members to acquire global barrier 2023-02-11 03:56:02,027 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-02-11 03:56:02,027 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:02,028 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:02,028 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:02,029 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:02,029 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-02-11 03:56:02,029 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-02-11 03:56:02,029 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,37011,1676087618428' in zk 2023-02-11 03:56:02,029 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:02,029 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 03:56:02,030 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-02-11 03:56:02,030 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:56:02,030 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:02,030 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-02-11 03:56:02,030 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:02,030 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:56:02,030 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:56:02,031 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:56:02,031 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:56:02,031 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:02,031 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:02,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:56:02,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:02,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:02,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,37011,1676087618428': 2023-02-11 03:56:02,033 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37011,1676087618428' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-02-11 03:56:02,033 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-02-11 03:56:02,033 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 03:56:02,033 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 03:56:02,033 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:02,033 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 03:56:02,036 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:02,036 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:56:02,036 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:02,036 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:02,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:02,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:02,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:56:02,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:56:02,036 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:56:02,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:56:02,036 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:02,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:56:02,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:02,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:02,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:56:02,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:02,038 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:02,038 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:02,038 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:56:02,038 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:02,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:02,040 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:02,040 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:02,040 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:02,040 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:02,040 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:56:02,040 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39921] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-02-11 03:56:02,040 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39921] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 03:56:02,040 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:56:02,040 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:56:02,040 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:56:02,040 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:02,040 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:02,040 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:02,040 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:56:02,040 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-02-11 03:56:02,040 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-02-11 03:56:02,041 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 03:56:02,041 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:56:02,041 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:56:02,141 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 03:56:02,142 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39921] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 03:56:02,146 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39921] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 03:56:02,148 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39921] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-02-11 03:56:02,149 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39921] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:02,149 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-02-11 03:56:02,149 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:56:02,149 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-02-11 03:56:02,149 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-02-11 03:56:02,149 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:02,149 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:02,151 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:02,151 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:02,151 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 03:56:02,152 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:56:02,152 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:56:02,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:56:02,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:02,153 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:02,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 03:56:02,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:02,153 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:02,157 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-02-11 03:56:02,157 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:02,157 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-02-11 03:56:02,157 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:56:02,157 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-02-11 03:56:02,157 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 03:56:02,157 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 03:56:02,157 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1676087757896.9c79e0360165a5fa04cbe6dbc395fafc. 2023-02-11 03:56:02,157 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1676087757896.9c79e0360165a5fa04cbe6dbc395fafc. started... 2023-02-11 03:56:02,157 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 9c79e0360165a5fa04cbe6dbc395fafc 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-11 03:56:02,173 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015120586bd20b0160a4666ba29d15afe99eba8 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e2015120586bd20b0160a4666ba29d15afe99eba8 2023-02-11 03:56:02,579 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMinorCompactionWithMonthlyPolicy/9c79e0360165a5fa04cbe6dbc395fafc/.tmp/family1/9a8f65eab8794ba1ac37142c620aa526 2023-02-11 03:56:02,583 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMinorCompactionWithMonthlyPolicy/9c79e0360165a5fa04cbe6dbc395fafc/.tmp/family1/9a8f65eab8794ba1ac37142c620aa526 as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMinorCompactionWithMonthlyPolicy/9c79e0360165a5fa04cbe6dbc395fafc/family1/9a8f65eab8794ba1ac37142c620aa526 2023-02-11 03:56:02,586 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMinorCompactionWithMonthlyPolicy/9c79e0360165a5fa04cbe6dbc395fafc/family1/9a8f65eab8794ba1ac37142c620aa526, entries=1, sequenceid=17, filesize=4.9 K 2023-02-11 03:56:02,587 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 9c79e0360165a5fa04cbe6dbc395fafc in 430ms, sequenceid=17, compaction requested=true 2023-02-11 03:56:02,587 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 9c79e0360165a5fa04cbe6dbc395fafc: 2023-02-11 03:56:02,587 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1676087757896.9c79e0360165a5fa04cbe6dbc395fafc. 2023-02-11 03:56:02,587 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 03:56:02,587 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 03:56:02,587 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:02,588 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-02-11 03:56:02,588 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37011,1676087618428' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-02-11 03:56:02,589 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:02,589 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:02,589 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:02,589 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:56:02,589 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:56:02,589 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:02,589 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 03:56:02,590 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:56:02,590 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:56:02,590 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:02,590 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:02,590 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:56:02,591 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37011,1676087618428' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-02-11 03:56:02,591 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6735b56f[Count = 0] remaining members to acquire global barrier 2023-02-11 03:56:02,591 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-02-11 03:56:02,591 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:02,593 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:02,593 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:02,593 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:02,593 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-02-11 03:56:02,593 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-02-11 03:56:02,593 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,37011,1676087618428' in zk 2023-02-11 03:56:02,593 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:02,593 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 03:56:02,594 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:02,594 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-02-11 03:56:02,594 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:02,594 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:56:02,594 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:56:02,594 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:56:02,594 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-02-11 03:56:02,595 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:56:02,595 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:56:02,595 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:02,595 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:02,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:56:02,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:02,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:02,597 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,37011,1676087618428': 2023-02-11 03:56:02,597 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37011,1676087618428' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-02-11 03:56:02,597 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-02-11 03:56:02,597 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 03:56:02,597 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 03:56:02,597 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:02,597 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 03:56:02,598 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:02,598 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:02,598 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:56:02,598 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:02,598 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:56:02,598 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:56:02,598 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:02,598 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:02,599 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:56:02,599 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:56:02,599 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:02,599 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:56:02,599 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:02,599 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:02,599 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:56:02,599 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:02,600 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:02,600 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:02,600 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:56:02,600 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:02,600 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:02,601 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:02,601 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:56:02,602 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:02,602 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:56:02,602 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-02-11 03:56:02,602 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39921] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-02-11 03:56:02,602 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39921] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 03:56:02,602 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:56:02,602 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:02,602 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:56:02,602 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:02,602 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:56:02,602 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:02,602 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-02-11 03:56:02,602 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:02,602 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 03:56:02,602 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:56:02,602 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:56:02,602 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:02,702 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 03:56:02,703 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39921] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 03:56:02,707 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39921] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 03:56:02,708 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39921] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-02-11 03:56:02,708 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39921] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:02,708 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-02-11 03:56:02,708 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:56:02,708 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-02-11 03:56:02,709 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-02-11 03:56:02,709 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:02,709 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:02,711 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:56:02,711 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:02,711 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:56:02,711 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:56:02,711 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:02,711 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 03:56:02,711 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:02,712 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:02,712 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 03:56:02,712 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:02,712 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:02,712 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-02-11 03:56:02,712 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:02,712 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-02-11 03:56:02,712 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:56:02,713 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-02-11 03:56:02,713 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 03:56:02,713 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 03:56:02,713 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1676087757896.9c79e0360165a5fa04cbe6dbc395fafc. 2023-02-11 03:56:02,713 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1676087757896.9c79e0360165a5fa04cbe6dbc395fafc. started... 2023-02-11 03:56:02,713 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 9c79e0360165a5fa04cbe6dbc395fafc 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-11 03:56:03,124 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015122847f9d5ea3d4241788f55793239d527b5 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e2015122847f9d5ea3d4241788f55793239d527b5 2023-02-11 03:56:03,530 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=21, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMinorCompactionWithMonthlyPolicy/9c79e0360165a5fa04cbe6dbc395fafc/.tmp/family1/b3ed452ae70d4bf2a6b68a059198b59f 2023-02-11 03:56:03,535 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMinorCompactionWithMonthlyPolicy/9c79e0360165a5fa04cbe6dbc395fafc/.tmp/family1/b3ed452ae70d4bf2a6b68a059198b59f as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMinorCompactionWithMonthlyPolicy/9c79e0360165a5fa04cbe6dbc395fafc/family1/b3ed452ae70d4bf2a6b68a059198b59f 2023-02-11 03:56:03,538 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMinorCompactionWithMonthlyPolicy/9c79e0360165a5fa04cbe6dbc395fafc/family1/b3ed452ae70d4bf2a6b68a059198b59f, entries=1, sequenceid=21, filesize=4.9 K 2023-02-11 03:56:03,539 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 9c79e0360165a5fa04cbe6dbc395fafc in 826ms, sequenceid=21, compaction requested=true 2023-02-11 03:56:03,539 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 9c79e0360165a5fa04cbe6dbc395fafc: 2023-02-11 03:56:03,539 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1676087757896.9c79e0360165a5fa04cbe6dbc395fafc. 2023-02-11 03:56:03,539 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 03:56:03,540 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 03:56:03,540 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:03,540 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-02-11 03:56:03,540 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37011,1676087618428' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-02-11 03:56:03,541 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:03,541 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:03,541 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:03,541 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:56:03,541 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:56:03,541 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:03,541 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 03:56:03,541 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:56:03,542 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:56:03,542 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:03,542 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:03,542 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:56:03,543 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37011,1676087618428' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-02-11 03:56:03,543 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@59bf0b6e[Count = 0] remaining members to acquire global barrier 2023-02-11 03:56:03,543 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-02-11 03:56:03,543 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:03,543 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:03,544 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:03,544 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:03,544 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-02-11 03:56:03,544 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:03,544 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 03:56:03,544 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-02-11 03:56:03,544 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,37011,1676087618428' in zk 2023-02-11 03:56:03,545 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-02-11 03:56:03,545 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:03,545 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:56:03,545 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:03,545 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:56:03,545 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-02-11 03:56:03,545 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:56:03,545 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:56:03,546 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:56:03,546 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:03,546 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:03,546 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:56:03,546 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:03,547 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:03,547 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,37011,1676087618428': 2023-02-11 03:56:03,547 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37011,1676087618428' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-02-11 03:56:03,547 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-02-11 03:56:03,547 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 03:56:03,547 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 03:56:03,547 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:03,547 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 03:56:03,549 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:03,549 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:03,549 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:03,550 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:56:03,550 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:56:03,550 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:03,549 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:56:03,550 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:03,550 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:56:03,550 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:03,550 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:56:03,550 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:56:03,550 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:03,550 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:03,550 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:56:03,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:03,551 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:03,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:03,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:56:03,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:03,552 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:03,558 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:03,558 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:03,558 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:03,558 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:03,558 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:56:03,558 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39921] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-02-11 03:56:03,558 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39921] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 03:56:03,558 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:03,558 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:56:03,558 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:03,558 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:56:03,558 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:56:03,558 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:56:03,558 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:03,558 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-02-11 03:56:03,558 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 03:56:03,559 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:56:03,559 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:56:03,659 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 03:56:03,659 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39921] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 03:56:03,663 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39921] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 03:56:03,664 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39921] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-02-11 03:56:03,665 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39921] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:03,665 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-02-11 03:56:03,665 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:56:03,665 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-02-11 03:56:03,665 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-02-11 03:56:03,665 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:03,665 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:03,666 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:03,666 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:56:03,666 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:56:03,666 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:56:03,666 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:03,666 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 03:56:03,667 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:03,667 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:03,667 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 03:56:03,667 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:03,667 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:03,667 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-02-11 03:56:03,667 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:03,667 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-02-11 03:56:03,667 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:56:03,668 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-02-11 03:56:03,668 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 03:56:03,668 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 03:56:03,668 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1676087757896.9c79e0360165a5fa04cbe6dbc395fafc. 2023-02-11 03:56:03,668 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1676087757896.9c79e0360165a5fa04cbe6dbc395fafc. started... 2023-02-11 03:56:03,668 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 9c79e0360165a5fa04cbe6dbc395fafc 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-11 03:56:04,079 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151231c185aab56e2a445f8db3fde7fd9f4b79 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151231c185aab56e2a445f8db3fde7fd9f4b79 2023-02-11 03:56:04,084 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=25, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMinorCompactionWithMonthlyPolicy/9c79e0360165a5fa04cbe6dbc395fafc/.tmp/family1/e668d09333294b77b462e1c1ad25bf42 2023-02-11 03:56:04,088 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMinorCompactionWithMonthlyPolicy/9c79e0360165a5fa04cbe6dbc395fafc/.tmp/family1/e668d09333294b77b462e1c1ad25bf42 as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMinorCompactionWithMonthlyPolicy/9c79e0360165a5fa04cbe6dbc395fafc/family1/e668d09333294b77b462e1c1ad25bf42 2023-02-11 03:56:04,091 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMinorCompactionWithMonthlyPolicy/9c79e0360165a5fa04cbe6dbc395fafc/family1/e668d09333294b77b462e1c1ad25bf42, entries=1, sequenceid=25, filesize=4.9 K 2023-02-11 03:56:04,092 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 9c79e0360165a5fa04cbe6dbc395fafc in 424ms, sequenceid=25, compaction requested=true 2023-02-11 03:56:04,092 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 9c79e0360165a5fa04cbe6dbc395fafc: 2023-02-11 03:56:04,092 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1676087757896.9c79e0360165a5fa04cbe6dbc395fafc. 2023-02-11 03:56:04,092 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 03:56:04,092 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 03:56:04,092 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:04,092 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-02-11 03:56:04,092 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37011,1676087618428' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-02-11 03:56:04,095 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:04,095 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:04,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:04,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:56:04,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:56:04,095 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:04,095 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 03:56:04,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:56:04,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:56:04,096 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:04,096 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:04,096 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:56:04,096 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37011,1676087618428' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-02-11 03:56:04,096 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@53caeee0[Count = 0] remaining members to acquire global barrier 2023-02-11 03:56:04,096 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-02-11 03:56:04,096 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:04,097 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:04,098 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:04,098 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:04,098 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-02-11 03:56:04,098 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-02-11 03:56:04,098 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,37011,1676087618428' in zk 2023-02-11 03:56:04,098 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:04,098 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 03:56:04,099 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:04,099 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-02-11 03:56:04,099 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:04,099 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:56:04,099 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:56:04,099 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:56:04,099 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-02-11 03:56:04,099 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:56:04,100 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:56:04,100 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:04,100 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:04,100 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:56:04,100 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:04,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:04,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,37011,1676087618428': 2023-02-11 03:56:04,101 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37011,1676087618428' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-02-11 03:56:04,101 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-02-11 03:56:04,101 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 03:56:04,101 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 03:56:04,101 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:04,101 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 03:56:04,103 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:04,103 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:04,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:04,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:56:04,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:56:04,103 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:04,103 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:56:04,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:04,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:56:04,103 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:04,103 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:56:04,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:56:04,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:04,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:04,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:56:04,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:04,104 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:04,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:04,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:56:04,105 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:04,105 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:04,107 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:04,107 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:04,107 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:56:04,107 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:04,107 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:56:04,107 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:04,107 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39921] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-02-11 03:56:04,107 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39921] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 03:56:04,107 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:56:04,107 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:56:04,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:56:04,107 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:04,107 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:04,107 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-02-11 03:56:04,107 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:04,108 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 03:56:04,108 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:56:04,108 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:56:04,208 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 03:56:04,209 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39921] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 03:56:04,212 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39921] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 03:56:04,213 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39921] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-02-11 03:56:04,213 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39921] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:04,214 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-02-11 03:56:04,214 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:56:04,214 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-02-11 03:56:04,214 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-02-11 03:56:04,214 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:04,214 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:04,215 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:04,215 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:56:04,215 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:56:04,215 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:56:04,215 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:04,215 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 03:56:04,216 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:04,216 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:04,216 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 03:56:04,216 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:04,216 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:04,216 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-02-11 03:56:04,217 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:04,217 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-02-11 03:56:04,217 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:56:04,217 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-02-11 03:56:04,217 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 03:56:04,217 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 03:56:04,217 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1676087757896.9c79e0360165a5fa04cbe6dbc395fafc. 2023-02-11 03:56:04,217 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1676087757896.9c79e0360165a5fa04cbe6dbc395fafc. started... 2023-02-11 03:56:04,217 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 9c79e0360165a5fa04cbe6dbc395fafc 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-11 03:56:04,571 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-11 03:56:04,628 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160101ea3601467643434c8052f4b5e195a761 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20160101ea3601467643434c8052f4b5e195a761 2023-02-11 03:56:05,033 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=29, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMinorCompactionWithMonthlyPolicy/9c79e0360165a5fa04cbe6dbc395fafc/.tmp/family1/e48384af575c4b30b6f1653456e80f1e 2023-02-11 03:56:05,037 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMinorCompactionWithMonthlyPolicy/9c79e0360165a5fa04cbe6dbc395fafc/.tmp/family1/e48384af575c4b30b6f1653456e80f1e as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMinorCompactionWithMonthlyPolicy/9c79e0360165a5fa04cbe6dbc395fafc/family1/e48384af575c4b30b6f1653456e80f1e 2023-02-11 03:56:05,040 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMinorCompactionWithMonthlyPolicy/9c79e0360165a5fa04cbe6dbc395fafc/family1/e48384af575c4b30b6f1653456e80f1e, entries=1, sequenceid=29, filesize=4.9 K 2023-02-11 03:56:05,041 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 9c79e0360165a5fa04cbe6dbc395fafc in 824ms, sequenceid=29, compaction requested=true 2023-02-11 03:56:05,041 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 9c79e0360165a5fa04cbe6dbc395fafc: 2023-02-11 03:56:05,041 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1676087757896.9c79e0360165a5fa04cbe6dbc395fafc. 2023-02-11 03:56:05,041 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 03:56:05,041 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 03:56:05,041 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:05,041 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-02-11 03:56:05,041 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37011,1676087618428' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-02-11 03:56:05,043 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:05,043 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:05,043 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:05,043 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:56:05,043 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:56:05,043 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:05,043 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 03:56:05,043 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:56:05,044 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:56:05,044 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:05,044 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:05,044 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:56:05,045 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37011,1676087618428' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-02-11 03:56:05,045 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2645a14d[Count = 0] remaining members to acquire global barrier 2023-02-11 03:56:05,045 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-02-11 03:56:05,045 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:05,046 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:05,046 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:05,046 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:05,046 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-02-11 03:56:05,046 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-02-11 03:56:05,046 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,37011,1676087618428' in zk 2023-02-11 03:56:05,046 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:05,046 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 03:56:05,048 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-02-11 03:56:05,048 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:05,048 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:56:05,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:05,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:56:05,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:56:05,048 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-02-11 03:56:05,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:56:05,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:56:05,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:05,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:05,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:56:05,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:05,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:05,051 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,37011,1676087618428': 2023-02-11 03:56:05,051 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37011,1676087618428' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-02-11 03:56:05,051 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-02-11 03:56:05,051 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 03:56:05,051 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 03:56:05,051 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:05,051 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 03:56:05,052 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:05,052 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:05,052 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:05,052 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:05,052 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:56:05,052 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:05,052 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:56:05,052 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:56:05,052 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:56:05,052 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:56:05,052 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:56:05,053 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:05,053 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:05,053 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:05,053 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:56:05,053 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:05,054 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:05,054 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:05,054 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:56:05,054 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:05,054 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:05,055 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:05,056 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:05,056 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:56:05,056 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:56:05,056 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:56:05,056 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:05,056 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:56:05,056 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:56:05,056 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39921] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-02-11 03:56:05,056 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39921] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 03:56:05,056 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:05,056 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:05,056 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:05,056 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:05,056 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:56:05,056 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:56:05,056 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-02-11 03:56:05,056 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 03:56:05,156 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 03:56:05,157 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39921] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 03:56:05,161 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39921] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 03:56:05,163 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39921] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-02-11 03:56:05,163 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39921] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:05,163 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-02-11 03:56:05,163 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:56:05,163 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-02-11 03:56:05,163 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-02-11 03:56:05,163 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:05,163 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:05,164 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:05,164 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:56:05,164 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:56:05,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:56:05,165 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:05,165 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 03:56:05,165 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:05,165 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:05,165 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 03:56:05,165 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:05,165 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:05,165 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-02-11 03:56:05,166 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:05,166 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-02-11 03:56:05,166 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:56:05,166 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-02-11 03:56:05,166 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 03:56:05,166 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 03:56:05,166 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1676087757896.9c79e0360165a5fa04cbe6dbc395fafc. 2023-02-11 03:56:05,166 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1676087757896.9c79e0360165a5fa04cbe6dbc395fafc. started... 2023-02-11 03:56:05,166 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 9c79e0360165a5fa04cbe6dbc395fafc 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-11 03:56:05,577 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2016010333e96452e0d542b4b9c220107b46bd5c to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e2016010333e96452e0d542b4b9c220107b46bd5c 2023-02-11 03:56:05,982 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=33, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMinorCompactionWithMonthlyPolicy/9c79e0360165a5fa04cbe6dbc395fafc/.tmp/family1/af2455b96bdb4e419ba6aef461137b9e 2023-02-11 03:56:05,985 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMinorCompactionWithMonthlyPolicy/9c79e0360165a5fa04cbe6dbc395fafc/.tmp/family1/af2455b96bdb4e419ba6aef461137b9e as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMinorCompactionWithMonthlyPolicy/9c79e0360165a5fa04cbe6dbc395fafc/family1/af2455b96bdb4e419ba6aef461137b9e 2023-02-11 03:56:05,988 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMinorCompactionWithMonthlyPolicy/9c79e0360165a5fa04cbe6dbc395fafc/family1/af2455b96bdb4e419ba6aef461137b9e, entries=1, sequenceid=33, filesize=4.9 K 2023-02-11 03:56:05,989 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 9c79e0360165a5fa04cbe6dbc395fafc in 823ms, sequenceid=33, compaction requested=true 2023-02-11 03:56:05,989 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 9c79e0360165a5fa04cbe6dbc395fafc: 2023-02-11 03:56:05,989 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1676087757896.9c79e0360165a5fa04cbe6dbc395fafc. 2023-02-11 03:56:05,989 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 03:56:05,989 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 03:56:05,989 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:05,989 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-02-11 03:56:05,989 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37011,1676087618428' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-02-11 03:56:05,992 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:05,992 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:05,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:05,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:56:05,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:56:05,992 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:05,992 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 03:56:05,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:56:05,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:56:05,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:05,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:05,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:56:05,993 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37011,1676087618428' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-02-11 03:56:05,993 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6c11e627[Count = 0] remaining members to acquire global barrier 2023-02-11 03:56:05,993 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-02-11 03:56:05,993 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:05,994 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:05,994 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:05,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:05,994 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-02-11 03:56:05,994 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-02-11 03:56:05,994 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,37011,1676087618428' in zk 2023-02-11 03:56:05,995 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:05,995 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 03:56:05,996 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-02-11 03:56:05,996 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:56:05,996 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:05,996 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-02-11 03:56:05,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:05,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:56:05,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:56:05,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:56:05,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:56:05,997 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:05,997 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:05,997 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:56:05,997 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:05,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:05,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,37011,1676087618428': 2023-02-11 03:56:05,998 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37011,1676087618428' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-02-11 03:56:05,998 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-02-11 03:56:05,998 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 03:56:05,998 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 03:56:05,998 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:05,998 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 03:56:06,000 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:06,000 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:06,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:06,000 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:06,000 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:56:06,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:06,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:56:06,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:56:06,000 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:06,000 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:56:06,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:56:06,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:56:06,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:06,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:06,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:56:06,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:06,001 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:06,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:06,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:56:06,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:06,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:06,009 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:06,009 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:56:06,009 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:56:06,009 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:56:06,009 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:56:06,009 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:56:06,009 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39921] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-02-11 03:56:06,009 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39921] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 03:56:06,009 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-02-11 03:56:06,009 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:06,009 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:56:06,009 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:56:06,009 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:06,010 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-02-11 03:56:06,010 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 03:56:06,010 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:06,010 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:06,010 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:06,010 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:06,110 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 03:56:06,111 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39921] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 03:56:06,114 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39921] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 03:56:06,116 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39921] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-02-11 03:56:06,116 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39921] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:06,116 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-02-11 03:56:06,116 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:56:06,116 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-02-11 03:56:06,116 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-02-11 03:56:06,117 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:06,117 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:06,118 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:06,118 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:56:06,118 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:56:06,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:56:06,118 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:06,118 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 03:56:06,119 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:06,119 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:06,119 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 03:56:06,119 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:06,119 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:06,119 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-02-11 03:56:06,119 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:06,119 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-02-11 03:56:06,119 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:56:06,119 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-02-11 03:56:06,120 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 03:56:06,120 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 03:56:06,120 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1676087757896.9c79e0360165a5fa04cbe6dbc395fafc. 2023-02-11 03:56:06,120 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1676087757896.9c79e0360165a5fa04cbe6dbc395fafc. started... 2023-02-11 03:56:06,120 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 9c79e0360165a5fa04cbe6dbc395fafc 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-11 03:56:06,531 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20150907394ffcad11fb4c90b7dd4bb355c11432 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20150907394ffcad11fb4c90b7dd4bb355c11432 2023-02-11 03:56:06,936 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=37, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMinorCompactionWithMonthlyPolicy/9c79e0360165a5fa04cbe6dbc395fafc/.tmp/family1/3117ec128e9f4da38aaa8a2f9327491c 2023-02-11 03:56:06,940 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMinorCompactionWithMonthlyPolicy/9c79e0360165a5fa04cbe6dbc395fafc/.tmp/family1/3117ec128e9f4da38aaa8a2f9327491c as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMinorCompactionWithMonthlyPolicy/9c79e0360165a5fa04cbe6dbc395fafc/family1/3117ec128e9f4da38aaa8a2f9327491c 2023-02-11 03:56:06,943 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMinorCompactionWithMonthlyPolicy/9c79e0360165a5fa04cbe6dbc395fafc/family1/3117ec128e9f4da38aaa8a2f9327491c, entries=1, sequenceid=37, filesize=4.9 K 2023-02-11 03:56:06,944 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 9c79e0360165a5fa04cbe6dbc395fafc in 824ms, sequenceid=37, compaction requested=true 2023-02-11 03:56:06,944 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 9c79e0360165a5fa04cbe6dbc395fafc: 2023-02-11 03:56:06,944 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1676087757896.9c79e0360165a5fa04cbe6dbc395fafc. 2023-02-11 03:56:06,944 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 03:56:06,944 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 03:56:06,944 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:06,944 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-02-11 03:56:06,944 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37011,1676087618428' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-02-11 03:56:06,946 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:06,946 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:06,946 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:06,946 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:56:06,946 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:56:06,946 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:06,946 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 03:56:06,946 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:56:06,946 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:56:06,946 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:06,947 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:06,947 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:56:06,947 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37011,1676087618428' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-02-11 03:56:06,947 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4dd474b0[Count = 0] remaining members to acquire global barrier 2023-02-11 03:56:06,947 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-02-11 03:56:06,947 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:06,948 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:06,948 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:06,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:06,948 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-02-11 03:56:06,948 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-02-11 03:56:06,948 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:06,948 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 03:56:06,948 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,37011,1676087618428' in zk 2023-02-11 03:56:06,951 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-02-11 03:56:06,951 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:56:06,951 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:06,951 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-02-11 03:56:06,951 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:06,951 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:56:06,951 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:56:06,951 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:56:06,951 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:56:06,951 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:06,952 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:06,952 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:56:06,952 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:06,952 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:06,953 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,37011,1676087618428': 2023-02-11 03:56:06,953 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37011,1676087618428' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-02-11 03:56:06,953 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-02-11 03:56:06,953 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 03:56:06,953 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 03:56:06,953 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:06,953 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 03:56:06,954 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:06,954 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:06,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:06,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:56:06,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:56:06,954 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:06,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:06,954 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:56:06,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:56:06,955 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:56:06,955 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:06,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:56:06,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:06,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:06,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:56:06,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:06,956 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:06,956 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:06,956 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:56:06,956 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:06,956 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:06,958 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:06,958 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:56:06,958 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:06,958 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39921] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-02-11 03:56:06,958 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39921] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 03:56:06,958 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:06,958 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:56:06,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:56:06,958 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:56:06,958 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:56:06,958 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:06,958 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-02-11 03:56:06,958 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:56:06,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:56:06,958 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 03:56:06,958 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:06,958 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:06,958 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:07,058 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 03:56:07,059 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39921] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 03:56:07,063 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39921] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 03:56:07,064 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39921] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-02-11 03:56:07,064 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39921] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:07,064 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-02-11 03:56:07,064 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:56:07,064 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-02-11 03:56:07,064 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-02-11 03:56:07,065 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:07,065 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:07,066 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:07,066 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:56:07,066 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:56:07,066 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:56:07,066 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:07,066 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 03:56:07,066 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:07,066 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:07,067 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 03:56:07,067 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:07,067 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:07,067 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-02-11 03:56:07,067 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:07,067 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-02-11 03:56:07,067 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:56:07,067 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-02-11 03:56:07,067 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-11 03:56:07,067 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 03:56:07,067 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1676087757896.9c79e0360165a5fa04cbe6dbc395fafc. 2023-02-11 03:56:07,067 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1676087757896.9c79e0360165a5fa04cbe6dbc395fafc. started... 2023-02-11 03:56:07,068 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 9c79e0360165a5fa04cbe6dbc395fafc 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-11 03:56:07,478 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201511200adb6ad36a8f4b72a3bcdca1683cfe0d to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201511200adb6ad36a8f4b72a3bcdca1683cfe0d 2023-02-11 03:56:07,883 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=41, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMinorCompactionWithMonthlyPolicy/9c79e0360165a5fa04cbe6dbc395fafc/.tmp/family1/bca9c63d19bd452cb63c7826f22a6a10 2023-02-11 03:56:07,887 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMinorCompactionWithMonthlyPolicy/9c79e0360165a5fa04cbe6dbc395fafc/.tmp/family1/bca9c63d19bd452cb63c7826f22a6a10 as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMinorCompactionWithMonthlyPolicy/9c79e0360165a5fa04cbe6dbc395fafc/family1/bca9c63d19bd452cb63c7826f22a6a10 2023-02-11 03:56:07,891 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMinorCompactionWithMonthlyPolicy/9c79e0360165a5fa04cbe6dbc395fafc/family1/bca9c63d19bd452cb63c7826f22a6a10, entries=1, sequenceid=41, filesize=4.9 K 2023-02-11 03:56:07,891 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 9c79e0360165a5fa04cbe6dbc395fafc in 823ms, sequenceid=41, compaction requested=true 2023-02-11 03:56:07,891 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 9c79e0360165a5fa04cbe6dbc395fafc: 2023-02-11 03:56:07,892 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1676087757896.9c79e0360165a5fa04cbe6dbc395fafc. 2023-02-11 03:56:07,892 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-11 03:56:07,892 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-11 03:56:07,892 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:07,892 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-02-11 03:56:07,892 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37011,1676087618428' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-02-11 03:56:07,894 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:07,894 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:07,894 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:07,894 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:56:07,895 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:56:07,895 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:07,895 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 03:56:07,895 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:56:07,895 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:56:07,895 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:07,895 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:07,896 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:56:07,896 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37011,1676087618428' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-02-11 03:56:07,896 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@496d2e04[Count = 0] remaining members to acquire global barrier 2023-02-11 03:56:07,896 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-02-11 03:56:07,896 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:07,897 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:07,897 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:07,897 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:07,897 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-02-11 03:56:07,897 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-02-11 03:56:07,897 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,37011,1676087618428' in zk 2023-02-11 03:56:07,897 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:07,897 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 03:56:07,899 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-02-11 03:56:07,899 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:07,899 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:56:07,899 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:07,899 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:56:07,899 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-02-11 03:56:07,899 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:56:07,899 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:56:07,899 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:56:07,899 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:07,900 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:07,900 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:56:07,900 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:07,900 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:07,901 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,37011,1676087618428': 2023-02-11 03:56:07,901 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37011,1676087618428' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-02-11 03:56:07,901 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-02-11 03:56:07,901 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 03:56:07,901 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 03:56:07,901 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:07,901 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 03:56:07,902 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:07,902 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:07,902 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:07,902 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:56:07,902 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:56:07,902 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:07,902 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:07,902 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:56:07,902 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:56:07,902 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:56:07,903 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:07,902 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:56:07,903 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:07,903 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:07,903 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:56:07,903 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:07,903 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:07,904 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:07,904 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:56:07,904 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:07,904 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:07,907 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:07,907 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:07,907 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:07,907 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:07,907 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:07,907 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39921] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-02-11 03:56:07,907 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:56:07,907 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-02-11 03:56:07,907 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:07,907 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:56:07,907 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:07,907 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:56:07,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:56:07,907 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:56:07,907 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39921] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 03:56:07,907 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:56:07,907 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-02-11 03:56:07,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:56:07,907 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 03:56:08,008 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 03:56:08,009 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39921] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 03:56:08,010 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=39921] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'}] of the table testMinorCompactionWithMonthlyPolicy 2023-02-11 03:56:08,011 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676087621935] procedure2.ProcedureExecutor(1029): Stored pid=115, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE 2023-02-11 03:56:08,012 DEBUG [PEWorker-3] locking.LockProcedure(309): LOCKED pid=115, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE 2023-02-11 03:56:08,012 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676087621935] mob.MobUtils(833): start MOB compaction of files for table='testMinorCompactionWithMonthlyPolicy', column='family1', allFiles=false, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-11 03:56:08,012 INFO [PEWorker-3] procedure2.TimeoutExecutorThread(81): ADDED pid=115, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1676088368012 2023-02-11 03:56:08,014 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676087621935] compactions.PartitionedMobCompactor(142): is allFiles: false 2023-02-11 03:56:08,015 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676087621935] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 10 selected files, and 0 irrelevant files table 'testMinorCompactionWithMonthlyPolicy' and column 'family1' 2023-02-11 03:56:08,015 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676087621935] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMinorCompactionWithMonthlyPolicy' column='family1' 2023-02-11 03:56:08,015 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676087621935] zookeeper.ReadOnlyZKClient(139): Connect 0x187963cb to 127.0.0.1:53383 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-11 03:56:08,019 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676087621935] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@56e23e8a, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-11 03:56:08,019 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMinorCompactionWithMonthlyPolicy' and column='family1' 2023-02-11 03:56:08,444 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201601036dc5a3507e274b81a14bd06df174cce5 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201601036dc5a3507e274b81a14bd06df174cce5 2023-02-11 03:56:08,448 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20160101 table='testMinorCompactionWithMonthlyPolicy' column='family1' 2023-02-11 03:56:08,467 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-11 03:56:08,469 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:34298, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-11 03:56:08,475 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-02-11 03:56:08,476 INFO [RS-EventLoopGroup-1-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:50336, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-02-11 03:56:08,484 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101/family1/818a97cd1b744b828e77feed48849ed2 first=Optional[r5] last=Optional[r6] 2023-02-11 03:56:08,484 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMinorCompactionWithMonthlyPolicy,,1676087757896.9c79e0360165a5fa04cbe6dbc395fafc., hostname=jenkins-hbase4.apache.org,37011,1676087618428, seqNum=2 for row with hfile group [{family1,hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101/family1/818a97cd1b744b828e77feed48849ed2}] 2023-02-11 03:56:08,488 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37011] regionserver.HStore(763): Validating hfile at hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101/family1/818a97cd1b744b828e77feed48849ed2 for inclusion in 9c79e0360165a5fa04cbe6dbc395fafc/family1 2023-02-11 03:56:08,490 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37011] regionserver.HStore(775): HFile bounds: first=r5 last=r6 2023-02-11 03:56:08,490 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37011] regionserver.HStore(777): Region bounds: first= last= 2023-02-11 03:56:08,490 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37011] regionserver.HRegion(2445): Flush status journal for 9c79e0360165a5fa04cbe6dbc395fafc: 2023-02-11 03:56:08,491 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37011] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101/family1/818a97cd1b744b828e77feed48849ed2 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/staging/jenkins__testMinorCompactionWithMonthlyPolicy__pahnrf4mf1rnip0maiorp04gqn0ib8ngt2fhr3q2cb9a5cqkmudl773pun1ikr6m/family1/818a97cd1b744b828e77feed48849ed2 2023-02-11 03:56:08,493 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37011] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/staging/jenkins__testMinorCompactionWithMonthlyPolicy__pahnrf4mf1rnip0maiorp04gqn0ib8ngt2fhr3q2cb9a5cqkmudl773pun1ikr6m/family1/818a97cd1b744b828e77feed48849ed2 as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMinorCompactionWithMonthlyPolicy/9c79e0360165a5fa04cbe6dbc395fafc/family1/dba66fcd672945b8a2accb847a1dda20_SeqId_44_ 2023-02-11 03:56:08,493 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37011] regionserver.HStore(849): Loaded HFile hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/staging/jenkins__testMinorCompactionWithMonthlyPolicy__pahnrf4mf1rnip0maiorp04gqn0ib8ngt2fhr3q2cb9a5cqkmudl773pun1ikr6m/family1/818a97cd1b744b828e77feed48849ed2 into 9c79e0360165a5fa04cbe6dbc395fafc/family1 as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMinorCompactionWithMonthlyPolicy/9c79e0360165a5fa04cbe6dbc395fafc/family1/dba66fcd672945b8a2accb847a1dda20_SeqId_44_ - updating store file list. 2023-02-11 03:56:08,496 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37011] regionserver.HStore(882): Loaded HFile hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMinorCompactionWithMonthlyPolicy/9c79e0360165a5fa04cbe6dbc395fafc/family1/dba66fcd672945b8a2accb847a1dda20_SeqId_44_ into 9c79e0360165a5fa04cbe6dbc395fafc/family1 2023-02-11 03:56:08,496 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37011] regionserver.HStore(855): Successfully loaded hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/staging/jenkins__testMinorCompactionWithMonthlyPolicy__pahnrf4mf1rnip0maiorp04gqn0ib8ngt2fhr3q2cb9a5cqkmudl773pun1ikr6m/family1/818a97cd1b744b828e77feed48849ed2 into 9c79e0360165a5fa04cbe6dbc395fafc/family1 (new location: hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMinorCompactionWithMonthlyPolicy/9c79e0360165a5fa04cbe6dbc395fafc/family1/dba66fcd672945b8a2accb847a1dda20_SeqId_44_) 2023-02-11 03:56:08,497 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37011] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/staging/jenkins__testMinorCompactionWithMonthlyPolicy__pahnrf4mf1rnip0maiorp04gqn0ib8ngt2fhr3q2cb9a5cqkmudl773pun1ikr6m/family1/818a97cd1b744b828e77feed48849ed2 2023-02-11 03:56:08,497 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37011] compactions.SortedCompactionPolicy(75): Selecting compaction from 11 store files, 0 compacting, 11 eligible, 16 blocking 2023-02-11 03:56:08,499 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37011] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 10 files of size 50903 starting at candidate #1 after considering 44 permutations with 44 in ratio 2023-02-11 03:56:08,499 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37011] regionserver.HStore(1912): 9c79e0360165a5fa04cbe6dbc395fafc/family1 is initiating minor compaction 2023-02-11 03:56:08,499 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37011] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@3de8c7f6; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-11 03:56:08,499 INFO [RS:0;jenkins-hbase4:37011-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 9c79e0360165a5fa04cbe6dbc395fafc/family1 in testMinorCompactionWithMonthlyPolicy,,1676087757896.9c79e0360165a5fa04cbe6dbc395fafc. 2023-02-11 03:56:08,499 INFO [RS:0;jenkins-hbase4:37011-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMinorCompactionWithMonthlyPolicy/9c79e0360165a5fa04cbe6dbc395fafc/family1/50c1fe16b4ef47f18b7acf254d4a325a, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMinorCompactionWithMonthlyPolicy/9c79e0360165a5fa04cbe6dbc395fafc/family1/c655cab8bd904a9cb5bad450397df19b, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMinorCompactionWithMonthlyPolicy/9c79e0360165a5fa04cbe6dbc395fafc/family1/9a8f65eab8794ba1ac37142c620aa526, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMinorCompactionWithMonthlyPolicy/9c79e0360165a5fa04cbe6dbc395fafc/family1/b3ed452ae70d4bf2a6b68a059198b59f, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMinorCompactionWithMonthlyPolicy/9c79e0360165a5fa04cbe6dbc395fafc/family1/e668d09333294b77b462e1c1ad25bf42, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMinorCompactionWithMonthlyPolicy/9c79e0360165a5fa04cbe6dbc395fafc/family1/e48384af575c4b30b6f1653456e80f1e, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMinorCompactionWithMonthlyPolicy/9c79e0360165a5fa04cbe6dbc395fafc/family1/af2455b96bdb4e419ba6aef461137b9e, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMinorCompactionWithMonthlyPolicy/9c79e0360165a5fa04cbe6dbc395fafc/family1/3117ec128e9f4da38aaa8a2f9327491c, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMinorCompactionWithMonthlyPolicy/9c79e0360165a5fa04cbe6dbc395fafc/family1/bca9c63d19bd452cb63c7826f22a6a10, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMinorCompactionWithMonthlyPolicy/9c79e0360165a5fa04cbe6dbc395fafc/family1/dba66fcd672945b8a2accb847a1dda20_SeqId_44_] into tmpdir=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMinorCompactionWithMonthlyPolicy/9c79e0360165a5fa04cbe6dbc395fafc/.tmp, totalSize=49.7 K 2023-02-11 03:56:08,499 DEBUG [RS:0;jenkins-hbase4:37011-shortCompactions-0] compactions.Compactor(207): Compacting 50c1fe16b4ef47f18b7acf254d4a325a, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=9 2023-02-11 03:56:08,499 DEBUG [RS:0;jenkins-hbase4:37011-shortCompactions-0] compactions.Compactor(207): Compacting c655cab8bd904a9cb5bad450397df19b, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=13 2023-02-11 03:56:08,499 DEBUG [RS:0;jenkins-hbase4:37011-shortCompactions-0] compactions.Compactor(207): Compacting 9a8f65eab8794ba1ac37142c620aa526, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=17 2023-02-11 03:56:08,499 DEBUG [RS:0;jenkins-hbase4:37011-shortCompactions-0] compactions.Compactor(207): Compacting b3ed452ae70d4bf2a6b68a059198b59f, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=21 2023-02-11 03:56:08,499 DEBUG [RS:0;jenkins-hbase4:37011-shortCompactions-0] compactions.Compactor(207): Compacting e668d09333294b77b462e1c1ad25bf42, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=25 2023-02-11 03:56:08,499 DEBUG [RS:0;jenkins-hbase4:37011-shortCompactions-0] compactions.Compactor(207): Compacting e48384af575c4b30b6f1653456e80f1e, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=29 2023-02-11 03:56:08,499 DEBUG [RS:0;jenkins-hbase4:37011-shortCompactions-0] compactions.Compactor(207): Compacting af2455b96bdb4e419ba6aef461137b9e, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=33 2023-02-11 03:56:08,500 DEBUG [RS:0;jenkins-hbase4:37011-shortCompactions-0] compactions.Compactor(207): Compacting 3117ec128e9f4da38aaa8a2f9327491c, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=37 2023-02-11 03:56:08,500 DEBUG [RS:0;jenkins-hbase4:37011-shortCompactions-0] compactions.Compactor(207): Compacting bca9c63d19bd452cb63c7826f22a6a10, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=41 2023-02-11 03:56:08,500 DEBUG [RS:0;jenkins-hbase4:37011-shortCompactions-0] compactions.Compactor(207): Compacting dba66fcd672945b8a2accb847a1dda20_SeqId_44_, keycount=2, bloomtype=ROW, size=5.2 K, encoding=NONE, compression=NONE, seqNum=45 2023-02-11 03:56:08,502 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20160101 table='testMinorCompactionWithMonthlyPolicy' column='family1' 2023-02-11 03:56:08,503 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-11 03:56:08,504 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20160101ea3601467643434c8052f4b5e195a761 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20160101ea3601467643434c8052f4b5e195a761 2023-02-11 03:56:08,505 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e2016010333e96452e0d542b4b9c220107b46bd5c to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e2016010333e96452e0d542b4b9c220107b46bd5c 2023-02-11 03:56:08,505 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMinorCompactionWithMonthlyPolicy 2 expired mob files are deleted 2023-02-11 03:56:08,505 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101' 2023-02-11 03:56:08,506 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 2 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMinorCompactionWithMonthlyPolicy' and column='family1' 2023-02-11 03:56:08,506 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMinorCompactionWithMonthlyPolicy' and column='family1' 2023-02-11 03:56:08,543 INFO [RS:0;jenkins-hbase4:37011-shortCompactions-0] throttle.PressureAwareThroughputController(145): 9c79e0360165a5fa04cbe6dbc395fafc#family1#compaction#165 average throughput is 0.34 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 57.69 MB/second 2023-02-11 03:56:08,968 DEBUG [RS:0;jenkins-hbase4:37011-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMinorCompactionWithMonthlyPolicy/9c79e0360165a5fa04cbe6dbc395fafc/.tmp/family1/fc3da3c8b4d9449fbea340128655e961 as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMinorCompactionWithMonthlyPolicy/9c79e0360165a5fa04cbe6dbc395fafc/family1/fc3da3c8b4d9449fbea340128655e961 2023-02-11 03:56:08,974 INFO [RS:0;jenkins-hbase4:37011-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 10 file(s) in 9c79e0360165a5fa04cbe6dbc395fafc/family1 of 9c79e0360165a5fa04cbe6dbc395fafc into fc3da3c8b4d9449fbea340128655e961(size=6.6 K), total size for store is 172.6 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-11 03:56:08,974 DEBUG [RS:0;jenkins-hbase4:37011-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 9c79e0360165a5fa04cbe6dbc395fafc: 2023-02-11 03:56:08,974 INFO [RS:0;jenkins-hbase4:37011-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMinorCompactionWithMonthlyPolicy,,1676087757896.9c79e0360165a5fa04cbe6dbc395fafc., storeName=family1, fileCount=10, fileSize=49.7 K (5.0 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 5.2 K), priority=2, time=1676087768499; duration=0sec 2023-02-11 03:56:08,974 DEBUG [RS:0;jenkins-hbase4:37011-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-11 03:56:09,050 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMinorCompactionWithMonthlyPolicy/9c79e0360165a5fa04cbe6dbc395fafc/family1/50c1fe16b4ef47f18b7acf254d4a325a, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMinorCompactionWithMonthlyPolicy/9c79e0360165a5fa04cbe6dbc395fafc/family1/c655cab8bd904a9cb5bad450397df19b, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMinorCompactionWithMonthlyPolicy/9c79e0360165a5fa04cbe6dbc395fafc/family1/9a8f65eab8794ba1ac37142c620aa526, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMinorCompactionWithMonthlyPolicy/9c79e0360165a5fa04cbe6dbc395fafc/family1/b3ed452ae70d4bf2a6b68a059198b59f, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMinorCompactionWithMonthlyPolicy/9c79e0360165a5fa04cbe6dbc395fafc/family1/e668d09333294b77b462e1c1ad25bf42, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMinorCompactionWithMonthlyPolicy/9c79e0360165a5fa04cbe6dbc395fafc/family1/e48384af575c4b30b6f1653456e80f1e, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMinorCompactionWithMonthlyPolicy/9c79e0360165a5fa04cbe6dbc395fafc/family1/af2455b96bdb4e419ba6aef461137b9e, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMinorCompactionWithMonthlyPolicy/9c79e0360165a5fa04cbe6dbc395fafc/family1/3117ec128e9f4da38aaa8a2f9327491c, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMinorCompactionWithMonthlyPolicy/9c79e0360165a5fa04cbe6dbc395fafc/family1/bca9c63d19bd452cb63c7826f22a6a10, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMinorCompactionWithMonthlyPolicy/9c79e0360165a5fa04cbe6dbc395fafc/family1/dba66fcd672945b8a2accb847a1dda20_SeqId_44_] to archive 2023-02-11 03:56:09,051 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-11 03:56:09,053 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMinorCompactionWithMonthlyPolicy/9c79e0360165a5fa04cbe6dbc395fafc/family1/50c1fe16b4ef47f18b7acf254d4a325a to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testMinorCompactionWithMonthlyPolicy/9c79e0360165a5fa04cbe6dbc395fafc/family1/50c1fe16b4ef47f18b7acf254d4a325a 2023-02-11 03:56:09,054 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMinorCompactionWithMonthlyPolicy/9c79e0360165a5fa04cbe6dbc395fafc/family1/c655cab8bd904a9cb5bad450397df19b to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testMinorCompactionWithMonthlyPolicy/9c79e0360165a5fa04cbe6dbc395fafc/family1/c655cab8bd904a9cb5bad450397df19b 2023-02-11 03:56:09,055 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMinorCompactionWithMonthlyPolicy/9c79e0360165a5fa04cbe6dbc395fafc/family1/9a8f65eab8794ba1ac37142c620aa526 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testMinorCompactionWithMonthlyPolicy/9c79e0360165a5fa04cbe6dbc395fafc/family1/9a8f65eab8794ba1ac37142c620aa526 2023-02-11 03:56:09,056 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMinorCompactionWithMonthlyPolicy/9c79e0360165a5fa04cbe6dbc395fafc/family1/b3ed452ae70d4bf2a6b68a059198b59f to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testMinorCompactionWithMonthlyPolicy/9c79e0360165a5fa04cbe6dbc395fafc/family1/b3ed452ae70d4bf2a6b68a059198b59f 2023-02-11 03:56:09,059 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMinorCompactionWithMonthlyPolicy/9c79e0360165a5fa04cbe6dbc395fafc/family1/e668d09333294b77b462e1c1ad25bf42 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testMinorCompactionWithMonthlyPolicy/9c79e0360165a5fa04cbe6dbc395fafc/family1/e668d09333294b77b462e1c1ad25bf42 2023-02-11 03:56:09,060 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMinorCompactionWithMonthlyPolicy/9c79e0360165a5fa04cbe6dbc395fafc/family1/e48384af575c4b30b6f1653456e80f1e to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testMinorCompactionWithMonthlyPolicy/9c79e0360165a5fa04cbe6dbc395fafc/family1/e48384af575c4b30b6f1653456e80f1e 2023-02-11 03:56:09,061 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMinorCompactionWithMonthlyPolicy/9c79e0360165a5fa04cbe6dbc395fafc/family1/af2455b96bdb4e419ba6aef461137b9e to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testMinorCompactionWithMonthlyPolicy/9c79e0360165a5fa04cbe6dbc395fafc/family1/af2455b96bdb4e419ba6aef461137b9e 2023-02-11 03:56:09,062 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMinorCompactionWithMonthlyPolicy/9c79e0360165a5fa04cbe6dbc395fafc/family1/3117ec128e9f4da38aaa8a2f9327491c to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testMinorCompactionWithMonthlyPolicy/9c79e0360165a5fa04cbe6dbc395fafc/family1/3117ec128e9f4da38aaa8a2f9327491c 2023-02-11 03:56:09,063 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMinorCompactionWithMonthlyPolicy/9c79e0360165a5fa04cbe6dbc395fafc/family1/bca9c63d19bd452cb63c7826f22a6a10 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testMinorCompactionWithMonthlyPolicy/9c79e0360165a5fa04cbe6dbc395fafc/family1/bca9c63d19bd452cb63c7826f22a6a10 2023-02-11 03:56:09,064 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMinorCompactionWithMonthlyPolicy/9c79e0360165a5fa04cbe6dbc395fafc/family1/dba66fcd672945b8a2accb847a1dda20_SeqId_44_ to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testMinorCompactionWithMonthlyPolicy/9c79e0360165a5fa04cbe6dbc395fafc/family1/dba66fcd672945b8a2accb847a1dda20_SeqId_44_ 2023-02-11 03:56:09,367 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151231f4d5b96d23164ebdbfd28e438a6530fe to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151231f4d5b96d23164ebdbfd28e438a6530fe 2023-02-11 03:56:09,368 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151201 table='testMinorCompactionWithMonthlyPolicy' column='family1' 2023-02-11 03:56:09,399 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201/family1/be486b088c6f40f7b468aa08ea6bf350 first=Optional[r1] last=Optional[r4] 2023-02-11 03:56:09,399 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMinorCompactionWithMonthlyPolicy,,1676087757896.9c79e0360165a5fa04cbe6dbc395fafc., hostname=jenkins-hbase4.apache.org,37011,1676087618428, seqNum=2 for row with hfile group [{family1,hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201/family1/be486b088c6f40f7b468aa08ea6bf350}] 2023-02-11 03:56:09,401 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37011] regionserver.HStore(763): Validating hfile at hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201/family1/be486b088c6f40f7b468aa08ea6bf350 for inclusion in 9c79e0360165a5fa04cbe6dbc395fafc/family1 2023-02-11 03:56:09,404 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37011] regionserver.HStore(775): HFile bounds: first=r1 last=r4 2023-02-11 03:56:09,404 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37011] regionserver.HStore(777): Region bounds: first= last= 2023-02-11 03:56:09,404 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37011] regionserver.HRegion(2445): Flush status journal for 9c79e0360165a5fa04cbe6dbc395fafc: 2023-02-11 03:56:09,405 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37011] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201/family1/be486b088c6f40f7b468aa08ea6bf350 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/staging/jenkins__testMinorCompactionWithMonthlyPolicy__i166tt9ngchvpdm41bkatrmc3mtd0bde31numgq1fusoqpftdfo5gf7jhauj8umk/family1/be486b088c6f40f7b468aa08ea6bf350 2023-02-11 03:56:09,407 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37011] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/staging/jenkins__testMinorCompactionWithMonthlyPolicy__i166tt9ngchvpdm41bkatrmc3mtd0bde31numgq1fusoqpftdfo5gf7jhauj8umk/family1/be486b088c6f40f7b468aa08ea6bf350 as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMinorCompactionWithMonthlyPolicy/9c79e0360165a5fa04cbe6dbc395fafc/family1/316cd17e1929479898ea4277232d994b_SeqId_47_ 2023-02-11 03:56:09,407 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37011] regionserver.HStore(849): Loaded HFile hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/staging/jenkins__testMinorCompactionWithMonthlyPolicy__i166tt9ngchvpdm41bkatrmc3mtd0bde31numgq1fusoqpftdfo5gf7jhauj8umk/family1/be486b088c6f40f7b468aa08ea6bf350 into 9c79e0360165a5fa04cbe6dbc395fafc/family1 as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMinorCompactionWithMonthlyPolicy/9c79e0360165a5fa04cbe6dbc395fafc/family1/316cd17e1929479898ea4277232d994b_SeqId_47_ - updating store file list. 2023-02-11 03:56:09,410 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37011] regionserver.HStore(882): Loaded HFile hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMinorCompactionWithMonthlyPolicy/9c79e0360165a5fa04cbe6dbc395fafc/family1/316cd17e1929479898ea4277232d994b_SeqId_47_ into 9c79e0360165a5fa04cbe6dbc395fafc/family1 2023-02-11 03:56:09,410 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37011] regionserver.HStore(855): Successfully loaded hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/staging/jenkins__testMinorCompactionWithMonthlyPolicy__i166tt9ngchvpdm41bkatrmc3mtd0bde31numgq1fusoqpftdfo5gf7jhauj8umk/family1/be486b088c6f40f7b468aa08ea6bf350 into 9c79e0360165a5fa04cbe6dbc395fafc/family1 (new location: hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMinorCompactionWithMonthlyPolicy/9c79e0360165a5fa04cbe6dbc395fafc/family1/316cd17e1929479898ea4277232d994b_SeqId_47_) 2023-02-11 03:56:09,411 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37011] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/staging/jenkins__testMinorCompactionWithMonthlyPolicy__i166tt9ngchvpdm41bkatrmc3mtd0bde31numgq1fusoqpftdfo5gf7jhauj8umk/family1/be486b088c6f40f7b468aa08ea6bf350 2023-02-11 03:56:09,411 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37011] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-02-11 03:56:09,412 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37011] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 182349 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-02-11 03:56:09,412 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37011] regionserver.HStore(1912): 9c79e0360165a5fa04cbe6dbc395fafc/family1 is initiating minor compaction (all files) 2023-02-11 03:56:09,412 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37011] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@57f18f17; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-11 03:56:09,412 INFO [RS:0;jenkins-hbase4:37011-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 9c79e0360165a5fa04cbe6dbc395fafc/family1 in testMinorCompactionWithMonthlyPolicy,,1676087757896.9c79e0360165a5fa04cbe6dbc395fafc. 2023-02-11 03:56:09,412 INFO [RS:0;jenkins-hbase4:37011-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMinorCompactionWithMonthlyPolicy/9c79e0360165a5fa04cbe6dbc395fafc/family1/436ff91765344ba6be250001c5672c69, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMinorCompactionWithMonthlyPolicy/9c79e0360165a5fa04cbe6dbc395fafc/family1/fc3da3c8b4d9449fbea340128655e961, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMinorCompactionWithMonthlyPolicy/9c79e0360165a5fa04cbe6dbc395fafc/family1/316cd17e1929479898ea4277232d994b_SeqId_47_] into tmpdir=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMinorCompactionWithMonthlyPolicy/9c79e0360165a5fa04cbe6dbc395fafc/.tmp, totalSize=178.1 K 2023-02-11 03:56:09,412 DEBUG [RS:0;jenkins-hbase4:37011-shortCompactions-0] compactions.Compactor(207): Compacting 436ff91765344ba6be250001c5672c69, keycount=1000, bloomtype=ROW, size=166.0 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1448878842014 2023-02-11 03:56:09,413 DEBUG [RS:0;jenkins-hbase4:37011-shortCompactions-0] compactions.Compactor(207): Compacting fc3da3c8b4d9449fbea340128655e961, keycount=9, bloomtype=ROW, size=6.6 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1441621242014 2023-02-11 03:56:09,413 DEBUG [RS:0;jenkins-hbase4:37011-shortCompactions-0] compactions.Compactor(207): Compacting 316cd17e1929479898ea4277232d994b_SeqId_47_, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=48, earliestPutTs=1448965242014 2023-02-11 03:56:09,414 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151201 table='testMinorCompactionWithMonthlyPolicy' column='family1' 2023-02-11 03:56:09,414 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-11 03:56:09,416 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e2015120152bb7fc8b7b24909a70ebe9c11054699 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e2015120152bb7fc8b7b24909a70ebe9c11054699 2023-02-11 03:56:09,418 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e2015120586bd20b0160a4666ba29d15afe99eba8 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e2015120586bd20b0160a4666ba29d15afe99eba8 2023-02-11 03:56:09,420 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e2015122847f9d5ea3d4241788f55793239d527b5 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e2015122847f9d5ea3d4241788f55793239d527b5 2023-02-11 03:56:09,421 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151231c185aab56e2a445f8db3fde7fd9f4b79 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151231c185aab56e2a445f8db3fde7fd9f4b79 2023-02-11 03:56:09,421 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMinorCompactionWithMonthlyPolicy 4 expired mob files are deleted 2023-02-11 03:56:09,421 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201' 2023-02-11 03:56:09,422 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMinorCompactionWithMonthlyPolicy' and column='family1' 2023-02-11 03:56:09,422 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMinorCompactionWithMonthlyPolicy' and column='family1' 2023-02-11 03:56:09,427 INFO [RS:0;jenkins-hbase4:37011-shortCompactions-0] throttle.PressureAwareThroughputController(145): 9c79e0360165a5fa04cbe6dbc395fafc#family1#compaction#166 average throughput is 25.93 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 57.69 MB/second 2023-02-11 03:56:09,866 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151130394c21100daa48069935d20d040f585d to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151130394c21100daa48069935d20d040f585d 2023-02-11 03:56:09,867 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMinorCompactionWithMonthlyPolicy' column='family1' 2023-02-11 03:56:09,889 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101/family1/af582d41440b46e79162cbff56bb964b first=Optional[r00] last=Optional[r8] 2023-02-11 03:56:09,889 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMinorCompactionWithMonthlyPolicy,,1676087757896.9c79e0360165a5fa04cbe6dbc395fafc., hostname=jenkins-hbase4.apache.org,37011,1676087618428, seqNum=2 for row with hfile group [{family1,hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101/family1/af582d41440b46e79162cbff56bb964b}] 2023-02-11 03:56:09,891 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37011] regionserver.HStore(763): Validating hfile at hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101/family1/af582d41440b46e79162cbff56bb964b for inclusion in 9c79e0360165a5fa04cbe6dbc395fafc/family1 2023-02-11 03:56:09,893 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37011] regionserver.HStore(775): HFile bounds: first=r00 last=r8 2023-02-11 03:56:09,893 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37011] regionserver.HStore(777): Region bounds: first= last= 2023-02-11 03:56:09,894 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37011] regionserver.HRegion(2445): Flush status journal for 9c79e0360165a5fa04cbe6dbc395fafc: 2023-02-11 03:56:09,894 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37011] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101/family1/af582d41440b46e79162cbff56bb964b to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/staging/jenkins__testMinorCompactionWithMonthlyPolicy__ok5kjkp3lse025u3ija5hak99v85j05h0hdjjhp6jsksnqv1cr4h1kbfi682ma0f/family1/af582d41440b46e79162cbff56bb964b 2023-02-11 03:56:09,896 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37011] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/staging/jenkins__testMinorCompactionWithMonthlyPolicy__ok5kjkp3lse025u3ija5hak99v85j05h0hdjjhp6jsksnqv1cr4h1kbfi682ma0f/family1/af582d41440b46e79162cbff56bb964b as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMinorCompactionWithMonthlyPolicy/9c79e0360165a5fa04cbe6dbc395fafc/family1/61749ea664ad4a0d955957555bc8efe5_SeqId_49_ 2023-02-11 03:56:09,896 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37011] regionserver.HStore(849): Loaded HFile hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/staging/jenkins__testMinorCompactionWithMonthlyPolicy__ok5kjkp3lse025u3ija5hak99v85j05h0hdjjhp6jsksnqv1cr4h1kbfi682ma0f/family1/af582d41440b46e79162cbff56bb964b into 9c79e0360165a5fa04cbe6dbc395fafc/family1 as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMinorCompactionWithMonthlyPolicy/9c79e0360165a5fa04cbe6dbc395fafc/family1/61749ea664ad4a0d955957555bc8efe5_SeqId_49_ - updating store file list. 2023-02-11 03:56:09,899 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37011] regionserver.HStore(882): Loaded HFile hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMinorCompactionWithMonthlyPolicy/9c79e0360165a5fa04cbe6dbc395fafc/family1/61749ea664ad4a0d955957555bc8efe5_SeqId_49_ into 9c79e0360165a5fa04cbe6dbc395fafc/family1 2023-02-11 03:56:09,899 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37011] regionserver.HStore(855): Successfully loaded hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/staging/jenkins__testMinorCompactionWithMonthlyPolicy__ok5kjkp3lse025u3ija5hak99v85j05h0hdjjhp6jsksnqv1cr4h1kbfi682ma0f/family1/af582d41440b46e79162cbff56bb964b into 9c79e0360165a5fa04cbe6dbc395fafc/family1 (new location: hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMinorCompactionWithMonthlyPolicy/9c79e0360165a5fa04cbe6dbc395fafc/family1/61749ea664ad4a0d955957555bc8efe5_SeqId_49_) 2023-02-11 03:56:09,900 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37011] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/staging/jenkins__testMinorCompactionWithMonthlyPolicy__ok5kjkp3lse025u3ija5hak99v85j05h0hdjjhp6jsksnqv1cr4h1kbfi682ma0f/family1/af582d41440b46e79162cbff56bb964b 2023-02-11 03:56:09,901 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMinorCompactionWithMonthlyPolicy' column='family1' 2023-02-11 03:56:09,902 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-11 03:56:09,903 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201511200adb6ad36a8f4b72a3bcdca1683cfe0d to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201511200adb6ad36a8f4b72a3bcdca1683cfe0d 2023-02-11 03:56:09,904 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e2015112805f86d62a18e43d9ab11bf6d4c610fc5 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e2015112805f86d62a18e43d9ab11bf6d4c610fc5 2023-02-11 03:56:09,905 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151130ff996190d2dd49eba541c38c5f7e8cc5 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151130ff996190d2dd49eba541c38c5f7e8cc5 2023-02-11 03:56:09,905 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMinorCompactionWithMonthlyPolicy 3 expired mob files are deleted 2023-02-11 03:56:09,905 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101' 2023-02-11 03:56:09,905 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 3 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMinorCompactionWithMonthlyPolicy' and column='family1' 2023-02-11 03:56:09,905 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMinorCompactionWithMonthlyPolicy' and column='family1' 2023-02-11 03:56:09,905 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMinorCompactionWithMonthlyPolicy' and column='family1' 2023-02-11 03:56:09,905 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676087621935] compactions.PartitionedMobCompactor(354): After compaction, there are 4 mob files. table='testMinorCompactionWithMonthlyPolicy' column='family1' 2023-02-11 03:56:09,905 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676087621935] mob.MobUtils(840): end MOB compaction of files for table='testMinorCompactionWithMonthlyPolicy', column='family1', allFiles=false, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-11 03:56:09,907 DEBUG [PEWorker-5] locking.LockProcedure(242): UNLOCKED pid=115, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE 2023-02-11 03:56:09,909 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=115, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE in 1.8970 sec 2023-02-11 03:56:10,028 INFO [Listener at localhost/46205] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMinorCompactionWithMonthlyPolicy Thread=256 (was 251) - Thread LEAK? -, OpenFileDescriptor=648 (was 626) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=129 (was 109) - SystemLoadAverage LEAK? -, ProcessCount=168 (was 168), AvailableMemoryMB=3151 (was 2865) - AvailableMemoryMB LEAK? - 2023-02-11 03:56:10,042 INFO [Listener at localhost/46205] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMajorCompactionFromAdmin Thread=256, OpenFileDescriptor=648, MaxFileDescriptor=60000, SystemLoadAverage=129, ProcessCount=168, AvailableMemoryMB=3151 2023-02-11 03:56:10,070 DEBUG [Listener at localhost/46205] crypto.Encryption(565): Installed org.apache.hadoop.hbase.io.crypto.KeyProviderForTesting into key provider cache 2023-02-11 03:56:10,123 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39921] master.HMaster$4(2102): Client=jenkins//172.31.14.131 create 'testMajorCompactionFromAdmin', {NAME => 'family1', ENCRYPTION => 'AES', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', ENCRYPTION_KEY => 'B\x0A\x03AES\x10\x10\x1A\x10\x04DH\xA4>\x17c~\xD3 \xA0\xBBE\xE4\x80j"\x10qhn3cY+H\x9D\x01]\x0E\xCD9\x81\xEE*\x10\xD8\x1C.\x9C\xC4\x8C3\x84et\x0E?\xEE\x8C_\xC42\x03MD5', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} 2023-02-11 03:56:10,124 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39921] procedure2.ProcedureExecutor(1029): Stored pid=116, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testMajorCompactionFromAdmin 2023-02-11 03:56:10,126 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=116, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionFromAdmin execute state=CREATE_TABLE_PRE_OPERATION 2023-02-11 03:56:10,126 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39921] master.MasterRpcServices(697): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testMajorCompactionFromAdmin" procId is: 116 2023-02-11 03:56:10,126 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=116, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testMajorCompactionFromAdmin execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-11 03:56:10,127 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39921] master.MasterRpcServices(1227): Checking to see if procedure is done pid=116 2023-02-11 03:56:10,128 DEBUG [HFileArchiver-20] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/.tmp/data/default/testMajorCompactionFromAdmin/953c5751d1228a55e688b51c4b8bb6fe 2023-02-11 03:56:10,128 DEBUG [HFileArchiver-21] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/.tmp/data/default/testMajorCompactionFromAdmin/fe3a2eacea8dd8daae28302db14142ed 2023-02-11 03:56:10,128 DEBUG [HFileArchiver-22] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/.tmp/data/default/testMajorCompactionFromAdmin/30759f6081f5ddc390558cd75a0299b8 2023-02-11 03:56:10,129 DEBUG [HFileArchiver-20] backup.HFileArchiver(153): Directory hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/.tmp/data/default/testMajorCompactionFromAdmin/953c5751d1228a55e688b51c4b8bb6fe empty. 2023-02-11 03:56:10,129 DEBUG [HFileArchiver-21] backup.HFileArchiver(153): Directory hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/.tmp/data/default/testMajorCompactionFromAdmin/fe3a2eacea8dd8daae28302db14142ed empty. 2023-02-11 03:56:10,129 DEBUG [HFileArchiver-22] backup.HFileArchiver(153): Directory hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/.tmp/data/default/testMajorCompactionFromAdmin/30759f6081f5ddc390558cd75a0299b8 empty. 2023-02-11 03:56:10,129 DEBUG [HFileArchiver-20] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/.tmp/data/default/testMajorCompactionFromAdmin/953c5751d1228a55e688b51c4b8bb6fe 2023-02-11 03:56:10,129 DEBUG [HFileArchiver-22] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/.tmp/data/default/testMajorCompactionFromAdmin/30759f6081f5ddc390558cd75a0299b8 2023-02-11 03:56:10,129 DEBUG [HFileArchiver-21] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/.tmp/data/default/testMajorCompactionFromAdmin/fe3a2eacea8dd8daae28302db14142ed 2023-02-11 03:56:10,129 DEBUG [PEWorker-2] procedure.DeleteTableProcedure(328): Archived testMajorCompactionFromAdmin regions 2023-02-11 03:56:10,227 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39921] master.MasterRpcServices(1227): Checking to see if procedure is done pid=116 2023-02-11 03:56:10,259 DEBUG [RS:0;jenkins-hbase4:37011-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMinorCompactionWithMonthlyPolicy/9c79e0360165a5fa04cbe6dbc395fafc/.tmp/family1/84dd76995758426c9610b32a82a95c76 as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMinorCompactionWithMonthlyPolicy/9c79e0360165a5fa04cbe6dbc395fafc/family1/84dd76995758426c9610b32a82a95c76 2023-02-11 03:56:10,263 INFO [RS:0;jenkins-hbase4:37011-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in 9c79e0360165a5fa04cbe6dbc395fafc/family1 of 9c79e0360165a5fa04cbe6dbc395fafc into 84dd76995758426c9610b32a82a95c76(size=167.4 K), total size for store is 333.6 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-11 03:56:10,263 DEBUG [RS:0;jenkins-hbase4:37011-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 9c79e0360165a5fa04cbe6dbc395fafc: 2023-02-11 03:56:10,263 INFO [RS:0;jenkins-hbase4:37011-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMinorCompactionWithMonthlyPolicy,,1676087757896.9c79e0360165a5fa04cbe6dbc395fafc., storeName=family1, fileCount=3, fileSize=178.1 K (166.0 K, 6.6 K, 5.5 K), priority=2, time=1676087769412; duration=0sec 2023-02-11 03:56:10,263 DEBUG [RS:0;jenkins-hbase4:37011-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-11 03:56:10,351 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMinorCompactionWithMonthlyPolicy/9c79e0360165a5fa04cbe6dbc395fafc/family1/436ff91765344ba6be250001c5672c69, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMinorCompactionWithMonthlyPolicy/9c79e0360165a5fa04cbe6dbc395fafc/family1/fc3da3c8b4d9449fbea340128655e961, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMinorCompactionWithMonthlyPolicy/9c79e0360165a5fa04cbe6dbc395fafc/family1/316cd17e1929479898ea4277232d994b_SeqId_47_] to archive 2023-02-11 03:56:10,352 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-11 03:56:10,354 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMinorCompactionWithMonthlyPolicy/9c79e0360165a5fa04cbe6dbc395fafc/family1/436ff91765344ba6be250001c5672c69 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testMinorCompactionWithMonthlyPolicy/9c79e0360165a5fa04cbe6dbc395fafc/family1/436ff91765344ba6be250001c5672c69 2023-02-11 03:56:10,355 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMinorCompactionWithMonthlyPolicy/9c79e0360165a5fa04cbe6dbc395fafc/family1/fc3da3c8b4d9449fbea340128655e961 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testMinorCompactionWithMonthlyPolicy/9c79e0360165a5fa04cbe6dbc395fafc/family1/fc3da3c8b4d9449fbea340128655e961 2023-02-11 03:56:10,356 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMinorCompactionWithMonthlyPolicy/9c79e0360165a5fa04cbe6dbc395fafc/family1/316cd17e1929479898ea4277232d994b_SeqId_47_ to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testMinorCompactionWithMonthlyPolicy/9c79e0360165a5fa04cbe6dbc395fafc/family1/316cd17e1929479898ea4277232d994b_SeqId_47_ 2023-02-11 03:56:10,428 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39921] master.MasterRpcServices(1227): Checking to see if procedure is done pid=116 2023-02-11 03:56:10,536 DEBUG [PEWorker-2] util.FSTableDescriptors(570): Wrote into hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/.tmp/data/default/testMajorCompactionFromAdmin/.tabledesc/.tableinfo.0000000001 2023-02-11 03:56:10,538 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(7671): creating {ENCODED => 30759f6081f5ddc390558cd75a0299b8, NAME => 'testMajorCompactionFromAdmin,2,1676087770094.30759f6081f5ddc390558cd75a0299b8.', STARTKEY => '2', ENDKEY => ''}, tableDescriptor='testMajorCompactionFromAdmin', {NAME => 'family1', ENCRYPTION => 'AES', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', ENCRYPTION_KEY => 'B\x0A\x03AES\x10\x10\x1A\x10\x04DH\xA4>\x17c~\xD3 \xA0\xBBE\xE4\x80j"\x10qhn3cY+H\x9D\x01]\x0E\xCD9\x81\xEE*\x10\xD8\x1C.\x9C\xC4\x8C3\x84et\x0E?\xEE\x8C_\xC42\x03MD5', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/.tmp 2023-02-11 03:56:10,538 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(7671): creating {ENCODED => fe3a2eacea8dd8daae28302db14142ed, NAME => 'testMajorCompactionFromAdmin,1,1676087770094.fe3a2eacea8dd8daae28302db14142ed.', STARTKEY => '1', ENDKEY => '2'}, tableDescriptor='testMajorCompactionFromAdmin', {NAME => 'family1', ENCRYPTION => 'AES', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', ENCRYPTION_KEY => 'B\x0A\x03AES\x10\x10\x1A\x10\x04DH\xA4>\x17c~\xD3 \xA0\xBBE\xE4\x80j"\x10qhn3cY+H\x9D\x01]\x0E\xCD9\x81\xEE*\x10\xD8\x1C.\x9C\xC4\x8C3\x84et\x0E?\xEE\x8C_\xC42\x03MD5', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/.tmp 2023-02-11 03:56:10,538 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(7671): creating {ENCODED => 953c5751d1228a55e688b51c4b8bb6fe, NAME => 'testMajorCompactionFromAdmin,,1676087770094.953c5751d1228a55e688b51c4b8bb6fe.', STARTKEY => '', ENDKEY => '1'}, tableDescriptor='testMajorCompactionFromAdmin', {NAME => 'family1', ENCRYPTION => 'AES', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', ENCRYPTION_KEY => 'B\x0A\x03AES\x10\x10\x1A\x10\x04DH\xA4>\x17c~\xD3 \xA0\xBBE\xE4\x80j"\x10qhn3cY+H\x9D\x01]\x0E\xCD9\x81\xEE*\x10\xD8\x1C.\x9C\xC4\x8C3\x84et\x0E?\xEE\x8C_\xC42\x03MD5', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/.tmp 2023-02-11 03:56:10,550 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(865): Instantiated testMajorCompactionFromAdmin,1,1676087770094.fe3a2eacea8dd8daae28302db14142ed.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-11 03:56:10,550 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(1603): Closing fe3a2eacea8dd8daae28302db14142ed, disabling compactions & flushes 2023-02-11 03:56:10,550 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(1625): Closing region testMajorCompactionFromAdmin,1,1676087770094.fe3a2eacea8dd8daae28302db14142ed. 2023-02-11 03:56:10,550 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(1646): Waiting without time limit for close lock on testMajorCompactionFromAdmin,1,1676087770094.fe3a2eacea8dd8daae28302db14142ed. 2023-02-11 03:56:10,550 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(1713): Acquired close lock on testMajorCompactionFromAdmin,1,1676087770094.fe3a2eacea8dd8daae28302db14142ed. after waiting 0 ms 2023-02-11 03:56:10,550 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(1723): Updates disabled for region testMajorCompactionFromAdmin,1,1676087770094.fe3a2eacea8dd8daae28302db14142ed. 2023-02-11 03:56:10,550 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(1837): Closed testMajorCompactionFromAdmin,1,1676087770094.fe3a2eacea8dd8daae28302db14142ed. 2023-02-11 03:56:10,550 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(1557): Region close journal for fe3a2eacea8dd8daae28302db14142ed: Waiting for close lock at 1676087770550Disabling compacts and flushes for region at 1676087770550Disabling writes for close at 1676087770550Writing region close event to WAL at 1676087770550Closed at 1676087770550 2023-02-11 03:56:10,729 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39921] master.MasterRpcServices(1227): Checking to see if procedure is done pid=116 2023-02-11 03:56:10,950 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(865): Instantiated testMajorCompactionFromAdmin,2,1676087770094.30759f6081f5ddc390558cd75a0299b8.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-11 03:56:10,950 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(1603): Closing 30759f6081f5ddc390558cd75a0299b8, disabling compactions & flushes 2023-02-11 03:56:10,950 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(1625): Closing region testMajorCompactionFromAdmin,2,1676087770094.30759f6081f5ddc390558cd75a0299b8. 2023-02-11 03:56:10,950 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(1646): Waiting without time limit for close lock on testMajorCompactionFromAdmin,2,1676087770094.30759f6081f5ddc390558cd75a0299b8. 2023-02-11 03:56:10,950 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(1713): Acquired close lock on testMajorCompactionFromAdmin,2,1676087770094.30759f6081f5ddc390558cd75a0299b8. after waiting 0 ms 2023-02-11 03:56:10,950 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(1723): Updates disabled for region testMajorCompactionFromAdmin,2,1676087770094.30759f6081f5ddc390558cd75a0299b8. 2023-02-11 03:56:10,950 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(1837): Closed testMajorCompactionFromAdmin,2,1676087770094.30759f6081f5ddc390558cd75a0299b8. 2023-02-11 03:56:10,950 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(1557): Region close journal for 30759f6081f5ddc390558cd75a0299b8: Waiting for close lock at 1676087770950Disabling compacts and flushes for region at 1676087770950Disabling writes for close at 1676087770950Writing region close event to WAL at 1676087770950Closed at 1676087770950 2023-02-11 03:56:10,950 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(865): Instantiated testMajorCompactionFromAdmin,,1676087770094.953c5751d1228a55e688b51c4b8bb6fe.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-11 03:56:10,951 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(1603): Closing 953c5751d1228a55e688b51c4b8bb6fe, disabling compactions & flushes 2023-02-11 03:56:10,951 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(1625): Closing region testMajorCompactionFromAdmin,,1676087770094.953c5751d1228a55e688b51c4b8bb6fe. 2023-02-11 03:56:10,951 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testMajorCompactionFromAdmin,,1676087770094.953c5751d1228a55e688b51c4b8bb6fe. 2023-02-11 03:56:10,951 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(1713): Acquired close lock on testMajorCompactionFromAdmin,,1676087770094.953c5751d1228a55e688b51c4b8bb6fe. after waiting 0 ms 2023-02-11 03:56:10,951 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(1723): Updates disabled for region testMajorCompactionFromAdmin,,1676087770094.953c5751d1228a55e688b51c4b8bb6fe. 2023-02-11 03:56:10,951 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(1837): Closed testMajorCompactionFromAdmin,,1676087770094.953c5751d1228a55e688b51c4b8bb6fe. 2023-02-11 03:56:10,951 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(1557): Region close journal for 953c5751d1228a55e688b51c4b8bb6fe: Waiting for close lock at 1676087770951Disabling compacts and flushes for region at 1676087770951Disabling writes for close at 1676087770951Writing region close event to WAL at 1676087770951Closed at 1676087770951 2023-02-11 03:56:10,953 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=116, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testMajorCompactionFromAdmin execute state=CREATE_TABLE_ADD_TO_META 2023-02-11 03:56:10,954 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMajorCompactionFromAdmin,1,1676087770094.fe3a2eacea8dd8daae28302db14142ed.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676087770954"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676087770954"}]},"ts":"1676087770954"} 2023-02-11 03:56:10,954 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMajorCompactionFromAdmin,2,1676087770094.30759f6081f5ddc390558cd75a0299b8.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1676087770954"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676087770954"}]},"ts":"1676087770954"} 2023-02-11 03:56:10,954 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMajorCompactionFromAdmin,,1676087770094.953c5751d1228a55e688b51c4b8bb6fe.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1676087770954"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676087770954"}]},"ts":"1676087770954"} 2023-02-11 03:56:10,956 INFO [PEWorker-2] hbase.MetaTableAccessor(1496): Added 3 regions to meta. 2023-02-11 03:56:10,956 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=116, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testMajorCompactionFromAdmin execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-11 03:56:10,956 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionFromAdmin","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676087770956"}]},"ts":"1676087770956"} 2023-02-11 03:56:10,957 INFO [PEWorker-2] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionFromAdmin, state=ENABLING in hbase:meta 2023-02-11 03:56:10,960 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=117, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=953c5751d1228a55e688b51c4b8bb6fe, ASSIGN}, {pid=118, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=fe3a2eacea8dd8daae28302db14142ed, ASSIGN}, {pid=119, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=30759f6081f5ddc390558cd75a0299b8, ASSIGN}] 2023-02-11 03:56:10,962 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=117, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=953c5751d1228a55e688b51c4b8bb6fe, ASSIGN 2023-02-11 03:56:10,962 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=118, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=fe3a2eacea8dd8daae28302db14142ed, ASSIGN 2023-02-11 03:56:10,962 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=119, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=30759f6081f5ddc390558cd75a0299b8, ASSIGN 2023-02-11 03:56:10,963 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=117, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=953c5751d1228a55e688b51c4b8bb6fe, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,37011,1676087618428; forceNewPlan=false, retain=false 2023-02-11 03:56:10,963 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=118, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=fe3a2eacea8dd8daae28302db14142ed, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,37011,1676087618428; forceNewPlan=false, retain=false 2023-02-11 03:56:10,963 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=119, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=30759f6081f5ddc390558cd75a0299b8, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,37011,1676087618428; forceNewPlan=false, retain=false 2023-02-11 03:56:11,114 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=117 updating hbase:meta row=953c5751d1228a55e688b51c4b8bb6fe, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:11,115 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=119 updating hbase:meta row=30759f6081f5ddc390558cd75a0299b8, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:11,115 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=118 updating hbase:meta row=fe3a2eacea8dd8daae28302db14142ed, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:11,115 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionFromAdmin,,1676087770094.953c5751d1228a55e688b51c4b8bb6fe.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1676087771114"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676087771114"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676087771114"}]},"ts":"1676087771114"} 2023-02-11 03:56:11,115 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionFromAdmin,2,1676087770094.30759f6081f5ddc390558cd75a0299b8.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1676087771114"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676087771114"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676087771114"}]},"ts":"1676087771114"} 2023-02-11 03:56:11,115 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionFromAdmin,1,1676087770094.fe3a2eacea8dd8daae28302db14142ed.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676087771114"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676087771114"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676087771114"}]},"ts":"1676087771114"} 2023-02-11 03:56:11,116 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=120, ppid=117, state=RUNNABLE; OpenRegionProcedure 953c5751d1228a55e688b51c4b8bb6fe, server=jenkins-hbase4.apache.org,37011,1676087618428}] 2023-02-11 03:56:11,117 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=121, ppid=118, state=RUNNABLE; OpenRegionProcedure fe3a2eacea8dd8daae28302db14142ed, server=jenkins-hbase4.apache.org,37011,1676087618428}] 2023-02-11 03:56:11,117 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=122, ppid=119, state=RUNNABLE; OpenRegionProcedure 30759f6081f5ddc390558cd75a0299b8, server=jenkins-hbase4.apache.org,37011,1676087618428}] 2023-02-11 03:56:11,230 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39921] master.MasterRpcServices(1227): Checking to see if procedure is done pid=116 2023-02-11 03:56:11,271 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionFromAdmin,1,1676087770094.fe3a2eacea8dd8daae28302db14142ed. 2023-02-11 03:56:11,271 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => fe3a2eacea8dd8daae28302db14142ed, NAME => 'testMajorCompactionFromAdmin,1,1676087770094.fe3a2eacea8dd8daae28302db14142ed.', STARTKEY => '1', ENDKEY => '2'} 2023-02-11 03:56:11,271 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionFromAdmin fe3a2eacea8dd8daae28302db14142ed 2023-02-11 03:56:11,271 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testMajorCompactionFromAdmin,1,1676087770094.fe3a2eacea8dd8daae28302db14142ed.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-11 03:56:11,271 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for fe3a2eacea8dd8daae28302db14142ed 2023-02-11 03:56:11,271 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for fe3a2eacea8dd8daae28302db14142ed 2023-02-11 03:56:11,292 INFO [StoreOpener-fe3a2eacea8dd8daae28302db14142ed-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region fe3a2eacea8dd8daae28302db14142ed 2023-02-11 03:56:11,294 DEBUG [StoreOpener-fe3a2eacea8dd8daae28302db14142ed-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/fe3a2eacea8dd8daae28302db14142ed/family1 2023-02-11 03:56:11,294 DEBUG [StoreOpener-fe3a2eacea8dd8daae28302db14142ed-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/fe3a2eacea8dd8daae28302db14142ed/family1 2023-02-11 03:56:11,294 INFO [StoreOpener-fe3a2eacea8dd8daae28302db14142ed-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region fe3a2eacea8dd8daae28302db14142ed columnFamilyName family1 2023-02-11 03:56:11,296 INFO [StoreOpener-fe3a2eacea8dd8daae28302db14142ed-1] regionserver.HStore(310): Store=fe3a2eacea8dd8daae28302db14142ed/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 03:56:11,296 INFO [StoreOpener-fe3a2eacea8dd8daae28302db14142ed-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region fe3a2eacea8dd8daae28302db14142ed 2023-02-11 03:56:11,298 DEBUG [StoreOpener-fe3a2eacea8dd8daae28302db14142ed-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/fe3a2eacea8dd8daae28302db14142ed/family2 2023-02-11 03:56:11,298 DEBUG [StoreOpener-fe3a2eacea8dd8daae28302db14142ed-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/fe3a2eacea8dd8daae28302db14142ed/family2 2023-02-11 03:56:11,299 INFO [StoreOpener-fe3a2eacea8dd8daae28302db14142ed-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region fe3a2eacea8dd8daae28302db14142ed columnFamilyName family2 2023-02-11 03:56:11,300 INFO [StoreOpener-fe3a2eacea8dd8daae28302db14142ed-1] regionserver.HStore(310): Store=fe3a2eacea8dd8daae28302db14142ed/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 03:56:11,301 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/fe3a2eacea8dd8daae28302db14142ed 2023-02-11 03:56:11,301 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/fe3a2eacea8dd8daae28302db14142ed 2023-02-11 03:56:11,303 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testMajorCompactionFromAdmin descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-11 03:56:11,303 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for fe3a2eacea8dd8daae28302db14142ed 2023-02-11 03:56:11,305 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/fe3a2eacea8dd8daae28302db14142ed/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-11 03:56:11,305 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened fe3a2eacea8dd8daae28302db14142ed; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=71439036, jitterRate=0.0645245909690857}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-11 03:56:11,305 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for fe3a2eacea8dd8daae28302db14142ed: Running coprocessor pre-open hook at 1676087771271Writing region info on filesystem at 1676087771271Initializing all the Stores at 1676087771272 (+1 ms)Instantiating store for column family {NAME => 'family1', ENCRYPTION => 'AES', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', ENCRYPTION_KEY => 'B\x0A\x03AES\x10\x10\x1A\x10\x04DH\xA4>\x17c~\xD3 \xA0\xBBE\xE4\x80j"\x10qhn3cY+H\x9D\x01]\x0E\xCD9\x81\xEE*\x10\xD8\x1C.\x9C\xC4\x8C3\x84et\x0E?\xEE\x8C_\xC42\x03MD5', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1676087771272Instantiating store for column family {NAME => 'family2', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1676087771272Cleaning up temporary data from old regions at 1676087771302 (+30 ms)Cleaning up detritus from prior splits at 1676087771302Running coprocessor post-open hooks at 1676087771305 (+3 ms)Region opened successfully at 1676087771305 2023-02-11 03:56:11,306 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionFromAdmin,1,1676087770094.fe3a2eacea8dd8daae28302db14142ed., pid=121, masterSystemTime=1676087771268 2023-02-11 03:56:11,308 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionFromAdmin,1,1676087770094.fe3a2eacea8dd8daae28302db14142ed. 2023-02-11 03:56:11,308 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionFromAdmin,1,1676087770094.fe3a2eacea8dd8daae28302db14142ed. 2023-02-11 03:56:11,308 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionFromAdmin,2,1676087770094.30759f6081f5ddc390558cd75a0299b8. 2023-02-11 03:56:11,308 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 30759f6081f5ddc390558cd75a0299b8, NAME => 'testMajorCompactionFromAdmin,2,1676087770094.30759f6081f5ddc390558cd75a0299b8.', STARTKEY => '2', ENDKEY => ''} 2023-02-11 03:56:11,308 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=118 updating hbase:meta row=fe3a2eacea8dd8daae28302db14142ed, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:11,308 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionFromAdmin 30759f6081f5ddc390558cd75a0299b8 2023-02-11 03:56:11,308 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionFromAdmin,1,1676087770094.fe3a2eacea8dd8daae28302db14142ed.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676087771308"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676087771308"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676087771308"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676087771308"}]},"ts":"1676087771308"} 2023-02-11 03:56:11,308 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testMajorCompactionFromAdmin,2,1676087770094.30759f6081f5ddc390558cd75a0299b8.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-11 03:56:11,308 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 30759f6081f5ddc390558cd75a0299b8 2023-02-11 03:56:11,308 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 30759f6081f5ddc390558cd75a0299b8 2023-02-11 03:56:11,311 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=121, resume processing ppid=118 2023-02-11 03:56:11,311 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=121, ppid=118, state=SUCCESS; OpenRegionProcedure fe3a2eacea8dd8daae28302db14142ed, server=jenkins-hbase4.apache.org,37011,1676087618428 in 192 msec 2023-02-11 03:56:11,312 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=118, ppid=116, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=fe3a2eacea8dd8daae28302db14142ed, ASSIGN in 351 msec 2023-02-11 03:56:11,329 INFO [StoreOpener-30759f6081f5ddc390558cd75a0299b8-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 30759f6081f5ddc390558cd75a0299b8 2023-02-11 03:56:11,330 DEBUG [StoreOpener-30759f6081f5ddc390558cd75a0299b8-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/30759f6081f5ddc390558cd75a0299b8/family1 2023-02-11 03:56:11,330 DEBUG [StoreOpener-30759f6081f5ddc390558cd75a0299b8-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/30759f6081f5ddc390558cd75a0299b8/family1 2023-02-11 03:56:11,331 INFO [StoreOpener-30759f6081f5ddc390558cd75a0299b8-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 30759f6081f5ddc390558cd75a0299b8 columnFamilyName family1 2023-02-11 03:56:11,332 INFO [StoreOpener-30759f6081f5ddc390558cd75a0299b8-1] regionserver.HStore(310): Store=30759f6081f5ddc390558cd75a0299b8/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 03:56:11,332 INFO [StoreOpener-30759f6081f5ddc390558cd75a0299b8-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 30759f6081f5ddc390558cd75a0299b8 2023-02-11 03:56:11,333 DEBUG [StoreOpener-30759f6081f5ddc390558cd75a0299b8-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/30759f6081f5ddc390558cd75a0299b8/family2 2023-02-11 03:56:11,333 DEBUG [StoreOpener-30759f6081f5ddc390558cd75a0299b8-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/30759f6081f5ddc390558cd75a0299b8/family2 2023-02-11 03:56:11,333 INFO [StoreOpener-30759f6081f5ddc390558cd75a0299b8-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 30759f6081f5ddc390558cd75a0299b8 columnFamilyName family2 2023-02-11 03:56:11,334 INFO [StoreOpener-30759f6081f5ddc390558cd75a0299b8-1] regionserver.HStore(310): Store=30759f6081f5ddc390558cd75a0299b8/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 03:56:11,335 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/30759f6081f5ddc390558cd75a0299b8 2023-02-11 03:56:11,335 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/30759f6081f5ddc390558cd75a0299b8 2023-02-11 03:56:11,336 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testMajorCompactionFromAdmin descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-11 03:56:11,337 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 30759f6081f5ddc390558cd75a0299b8 2023-02-11 03:56:11,339 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/30759f6081f5ddc390558cd75a0299b8/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-11 03:56:11,339 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 30759f6081f5ddc390558cd75a0299b8; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=67771631, jitterRate=0.00987599790096283}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-11 03:56:11,339 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 30759f6081f5ddc390558cd75a0299b8: Running coprocessor pre-open hook at 1676087771309Writing region info on filesystem at 1676087771309Initializing all the Stores at 1676087771309Instantiating store for column family {NAME => 'family1', ENCRYPTION => 'AES', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', ENCRYPTION_KEY => 'B\x0A\x03AES\x10\x10\x1A\x10\x04DH\xA4>\x17c~\xD3 \xA0\xBBE\xE4\x80j"\x10qhn3cY+H\x9D\x01]\x0E\xCD9\x81\xEE*\x10\xD8\x1C.\x9C\xC4\x8C3\x84et\x0E?\xEE\x8C_\xC42\x03MD5', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1676087771309Instantiating store for column family {NAME => 'family2', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1676087771309Cleaning up temporary data from old regions at 1676087771336 (+27 ms)Cleaning up detritus from prior splits at 1676087771336Running coprocessor post-open hooks at 1676087771339 (+3 ms)Region opened successfully at 1676087771339 2023-02-11 03:56:11,340 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionFromAdmin,2,1676087770094.30759f6081f5ddc390558cd75a0299b8., pid=122, masterSystemTime=1676087771268 2023-02-11 03:56:11,341 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionFromAdmin,2,1676087770094.30759f6081f5ddc390558cd75a0299b8. 2023-02-11 03:56:11,341 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionFromAdmin,2,1676087770094.30759f6081f5ddc390558cd75a0299b8. 2023-02-11 03:56:11,341 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionFromAdmin,,1676087770094.953c5751d1228a55e688b51c4b8bb6fe. 2023-02-11 03:56:11,342 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 953c5751d1228a55e688b51c4b8bb6fe, NAME => 'testMajorCompactionFromAdmin,,1676087770094.953c5751d1228a55e688b51c4b8bb6fe.', STARTKEY => '', ENDKEY => '1'} 2023-02-11 03:56:11,342 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=119 updating hbase:meta row=30759f6081f5ddc390558cd75a0299b8, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:11,342 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionFromAdmin 953c5751d1228a55e688b51c4b8bb6fe 2023-02-11 03:56:11,342 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testMajorCompactionFromAdmin,,1676087770094.953c5751d1228a55e688b51c4b8bb6fe.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-11 03:56:11,342 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionFromAdmin,2,1676087770094.30759f6081f5ddc390558cd75a0299b8.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1676087771342"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676087771342"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676087771342"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676087771342"}]},"ts":"1676087771342"} 2023-02-11 03:56:11,342 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 953c5751d1228a55e688b51c4b8bb6fe 2023-02-11 03:56:11,342 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 953c5751d1228a55e688b51c4b8bb6fe 2023-02-11 03:56:11,344 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=122, resume processing ppid=119 2023-02-11 03:56:11,344 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=122, ppid=119, state=SUCCESS; OpenRegionProcedure 30759f6081f5ddc390558cd75a0299b8, server=jenkins-hbase4.apache.org,37011,1676087618428 in 226 msec 2023-02-11 03:56:11,345 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=119, ppid=116, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=30759f6081f5ddc390558cd75a0299b8, ASSIGN in 384 msec 2023-02-11 03:56:11,364 INFO [StoreOpener-953c5751d1228a55e688b51c4b8bb6fe-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 953c5751d1228a55e688b51c4b8bb6fe 2023-02-11 03:56:11,365 DEBUG [StoreOpener-953c5751d1228a55e688b51c4b8bb6fe-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/953c5751d1228a55e688b51c4b8bb6fe/family1 2023-02-11 03:56:11,365 DEBUG [StoreOpener-953c5751d1228a55e688b51c4b8bb6fe-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/953c5751d1228a55e688b51c4b8bb6fe/family1 2023-02-11 03:56:11,365 INFO [StoreOpener-953c5751d1228a55e688b51c4b8bb6fe-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 953c5751d1228a55e688b51c4b8bb6fe columnFamilyName family1 2023-02-11 03:56:11,366 INFO [StoreOpener-953c5751d1228a55e688b51c4b8bb6fe-1] regionserver.HStore(310): Store=953c5751d1228a55e688b51c4b8bb6fe/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 03:56:11,366 INFO [StoreOpener-953c5751d1228a55e688b51c4b8bb6fe-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 953c5751d1228a55e688b51c4b8bb6fe 2023-02-11 03:56:11,367 DEBUG [StoreOpener-953c5751d1228a55e688b51c4b8bb6fe-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/953c5751d1228a55e688b51c4b8bb6fe/family2 2023-02-11 03:56:11,367 DEBUG [StoreOpener-953c5751d1228a55e688b51c4b8bb6fe-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/953c5751d1228a55e688b51c4b8bb6fe/family2 2023-02-11 03:56:11,367 INFO [StoreOpener-953c5751d1228a55e688b51c4b8bb6fe-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 953c5751d1228a55e688b51c4b8bb6fe columnFamilyName family2 2023-02-11 03:56:11,368 INFO [StoreOpener-953c5751d1228a55e688b51c4b8bb6fe-1] regionserver.HStore(310): Store=953c5751d1228a55e688b51c4b8bb6fe/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-11 03:56:11,369 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/953c5751d1228a55e688b51c4b8bb6fe 2023-02-11 03:56:11,369 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/953c5751d1228a55e688b51c4b8bb6fe 2023-02-11 03:56:11,371 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testMajorCompactionFromAdmin descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-11 03:56:11,372 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 953c5751d1228a55e688b51c4b8bb6fe 2023-02-11 03:56:11,373 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/953c5751d1228a55e688b51c4b8bb6fe/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-11 03:56:11,373 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 953c5751d1228a55e688b51c4b8bb6fe; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=67425732, jitterRate=0.004721701145172119}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-11 03:56:11,373 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 953c5751d1228a55e688b51c4b8bb6fe: Running coprocessor pre-open hook at 1676087771342Writing region info on filesystem at 1676087771342Initializing all the Stores at 1676087771343 (+1 ms)Instantiating store for column family {NAME => 'family1', ENCRYPTION => 'AES', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', ENCRYPTION_KEY => 'B\x0A\x03AES\x10\x10\x1A\x10\x04DH\xA4>\x17c~\xD3 \xA0\xBBE\xE4\x80j"\x10qhn3cY+H\x9D\x01]\x0E\xCD9\x81\xEE*\x10\xD8\x1C.\x9C\xC4\x8C3\x84et\x0E?\xEE\x8C_\xC42\x03MD5', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1676087771343Instantiating store for column family {NAME => 'family2', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1676087771343Cleaning up temporary data from old regions at 1676087771369 (+26 ms)Cleaning up detritus from prior splits at 1676087771369Running coprocessor post-open hooks at 1676087771373 (+4 ms)Region opened successfully at 1676087771373 2023-02-11 03:56:11,374 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionFromAdmin,,1676087770094.953c5751d1228a55e688b51c4b8bb6fe., pid=120, masterSystemTime=1676087771268 2023-02-11 03:56:11,375 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionFromAdmin,,1676087770094.953c5751d1228a55e688b51c4b8bb6fe. 2023-02-11 03:56:11,375 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionFromAdmin,,1676087770094.953c5751d1228a55e688b51c4b8bb6fe. 2023-02-11 03:56:11,376 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=117 updating hbase:meta row=953c5751d1228a55e688b51c4b8bb6fe, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:11,376 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionFromAdmin,,1676087770094.953c5751d1228a55e688b51c4b8bb6fe.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1676087771376"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676087771376"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676087771376"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676087771376"}]},"ts":"1676087771376"} 2023-02-11 03:56:11,378 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=120, resume processing ppid=117 2023-02-11 03:56:11,378 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=120, ppid=117, state=SUCCESS; OpenRegionProcedure 953c5751d1228a55e688b51c4b8bb6fe, server=jenkins-hbase4.apache.org,37011,1676087618428 in 261 msec 2023-02-11 03:56:11,379 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=117, resume processing ppid=116 2023-02-11 03:56:11,379 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=117, ppid=116, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=953c5751d1228a55e688b51c4b8bb6fe, ASSIGN in 418 msec 2023-02-11 03:56:11,380 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=116, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testMajorCompactionFromAdmin execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-11 03:56:11,380 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionFromAdmin","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676087771380"}]},"ts":"1676087771380"} 2023-02-11 03:56:11,381 INFO [PEWorker-2] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionFromAdmin, state=ENABLED in hbase:meta 2023-02-11 03:56:11,383 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=116, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionFromAdmin execute state=CREATE_TABLE_POST_OPERATION 2023-02-11 03:56:11,384 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=116, state=SUCCESS; CreateTableProcedure table=testMajorCompactionFromAdmin in 1.2600 sec 2023-02-11 03:56:12,231 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39921] master.MasterRpcServices(1227): Checking to see if procedure is done pid=116 2023-02-11 03:56:12,231 INFO [Listener at localhost/46205] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testMajorCompactionFromAdmin, procId: 116 completed 2023-02-11 03:56:12,237 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37011] regionserver.HRegion(8801): writing data to region testMajorCompactionFromAdmin,,1676087770094.953c5751d1228a55e688b51c4b8bb6fe. with WAL disabled. Data may be lost in the event of a crash. 2023-02-11 03:56:12,238 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37011] regionserver.HRegion(8801): writing data to region testMajorCompactionFromAdmin,1,1676087770094.fe3a2eacea8dd8daae28302db14142ed. with WAL disabled. Data may be lost in the event of a crash. 2023-02-11 03:56:12,238 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37011] regionserver.HRegion(8801): writing data to region testMajorCompactionFromAdmin,2,1676087770094.30759f6081f5ddc390558cd75a0299b8. with WAL disabled. Data may be lost in the event of a crash. 2023-02-11 03:56:12,240 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39921] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 03:56:12,241 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39921] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionFromAdmin 2023-02-11 03:56:12,241 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionFromAdmin' 2023-02-11 03:56:12,241 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:56:12,241 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionFromAdmin' starting 'acquire' 2023-02-11 03:56:12,241 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionFromAdmin', kicking off acquire phase on members. 2023-02-11 03:56:12,242 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-11 03:56:12,242 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-02-11 03:56:12,243 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:12,243 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:56:12,243 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:56:12,243 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:56:12,243 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:12,243 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 03:56:12,243 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-02-11 03:56:12,244 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-11 03:56:12,244 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 03:56:12,244 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-02-11 03:56:12,244 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionFromAdmin 2023-02-11 03:56:12,244 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionFromAdmin 2023-02-11 03:56:12,244 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionFromAdmin' with timeout 60000ms 2023-02-11 03:56:12,244 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:56:12,244 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionFromAdmin' starting 'acquire' stage 2023-02-11 03:56:12,245 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,1,1676087770094.fe3a2eacea8dd8daae28302db14142ed. 2023-02-11 03:56:12,245 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-02-11 03:56:12,245 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 03:56:12,245 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,,1676087770094.953c5751d1228a55e688b51c4b8bb6fe. 2023-02-11 03:56:12,245 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,,1676087770094.953c5751d1228a55e688b51c4b8bb6fe. started... 2023-02-11 03:56:12,245 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,1,1676087770094.fe3a2eacea8dd8daae28302db14142ed. started... 2023-02-11 03:56:12,245 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,2,1676087770094.30759f6081f5ddc390558cd75a0299b8. 2023-02-11 03:56:12,246 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,2,1676087770094.30759f6081f5ddc390558cd75a0299b8. started... 2023-02-11 03:56:12,246 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-1] regionserver.HRegion(2744): Flushing 953c5751d1228a55e688b51c4b8bb6fe 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-02-11 03:56:12,246 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing fe3a2eacea8dd8daae28302db14142ed 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-02-11 03:56:12,246 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-2] regionserver.HRegion(2744): Flushing 30759f6081f5ddc390558cd75a0299b8 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-02-11 03:56:12,689 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202302113342da04ff594f2ab9814585df6a50d0 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b202302113342da04ff594f2ab9814585df6a50d0 2023-02-11 03:56:12,689 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202302115b31200b3df7470a889738d32b801104 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c202302115b31200b3df7470a889738d32b801104 2023-02-11 03:56:12,691 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202302117b9b517990204390bbd2619a48d244b9 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e202302117b9b517990204390bbd2619a48d244b9 2023-02-11 03:56:12,704 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/fe3a2eacea8dd8daae28302db14142ed/.tmp/family1/e7cf10565b304f34b02db69e5b8863cc 2023-02-11 03:56:12,705 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/953c5751d1228a55e688b51c4b8bb6fe/.tmp/family1/9bcd485d965341ed8dab23d2ff57c13d 2023-02-11 03:56:12,745 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202302118c75502945d74618b124c940da34c745 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/d41d8cd98f00b204e9800998ecf8427e202302118c75502945d74618b124c940da34c745 2023-02-11 03:56:13,110 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/30759f6081f5ddc390558cd75a0299b8/.tmp/family1/394aad0fbd074d8093fb08363fb9a75f 2023-02-11 03:56:13,144 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230211e3cddf7a86914a38b18b471abf223d90 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c4ca4238a0b923820dcc509a6f75849b20230211e3cddf7a86914a38b18b471abf223d90 2023-02-11 03:56:13,149 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/953c5751d1228a55e688b51c4b8bb6fe/.tmp/family2/f7e13c216d95421c832d580d536a91b4 2023-02-11 03:56:13,152 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/953c5751d1228a55e688b51c4b8bb6fe/.tmp/family1/9bcd485d965341ed8dab23d2ff57c13d as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/953c5751d1228a55e688b51c4b8bb6fe/family1/9bcd485d965341ed8dab23d2ff57c13d 2023-02-11 03:56:13,175 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/953c5751d1228a55e688b51c4b8bb6fe/family1/9bcd485d965341ed8dab23d2ff57c13d, entries=4, sequenceid=5, filesize=5.5 K 2023-02-11 03:56:13,175 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/953c5751d1228a55e688b51c4b8bb6fe/.tmp/family2/f7e13c216d95421c832d580d536a91b4 as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/953c5751d1228a55e688b51c4b8bb6fe/family2/f7e13c216d95421c832d580d536a91b4 2023-02-11 03:56:13,178 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/953c5751d1228a55e688b51c4b8bb6fe/family2/f7e13c216d95421c832d580d536a91b4, entries=2, sequenceid=5, filesize=5.1 K 2023-02-11 03:56:13,179 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-1] regionserver.HRegion(2947): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for 953c5751d1228a55e688b51c4b8bb6fe in 933ms, sequenceid=5, compaction requested=false 2023-02-11 03:56:13,179 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testMajorCompactionFromAdmin' 2023-02-11 03:56:13,180 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-1] regionserver.HRegion(2445): Flush status journal for 953c5751d1228a55e688b51c4b8bb6fe: 2023-02-11 03:56:13,180 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,,1676087770094.953c5751d1228a55e688b51c4b8bb6fe. 2023-02-11 03:56:13,180 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-02-11 03:56:13,549 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/fe3a2eacea8dd8daae28302db14142ed/.tmp/family2/0362b086d5c94dec90e6cd8d7edeedcf 2023-02-11 03:56:13,552 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230211a18898e2a2c243c4966860952c45a393 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c81e728d9d4c2f636f067f89cc14862c20230211a18898e2a2c243c4966860952c45a393 2023-02-11 03:56:13,557 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/fe3a2eacea8dd8daae28302db14142ed/.tmp/family1/e7cf10565b304f34b02db69e5b8863cc as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/fe3a2eacea8dd8daae28302db14142ed/family1/e7cf10565b304f34b02db69e5b8863cc 2023-02-11 03:56:13,581 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/fe3a2eacea8dd8daae28302db14142ed/family1/e7cf10565b304f34b02db69e5b8863cc, entries=4, sequenceid=5, filesize=5.5 K 2023-02-11 03:56:13,582 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/fe3a2eacea8dd8daae28302db14142ed/.tmp/family2/0362b086d5c94dec90e6cd8d7edeedcf as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/fe3a2eacea8dd8daae28302db14142ed/family2/0362b086d5c94dec90e6cd8d7edeedcf 2023-02-11 03:56:13,585 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/fe3a2eacea8dd8daae28302db14142ed/family2/0362b086d5c94dec90e6cd8d7edeedcf, entries=2, sequenceid=5, filesize=5.1 K 2023-02-11 03:56:13,586 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for fe3a2eacea8dd8daae28302db14142ed in 1340ms, sequenceid=5, compaction requested=false 2023-02-11 03:56:13,587 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for fe3a2eacea8dd8daae28302db14142ed: 2023-02-11 03:56:13,587 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,1,1676087770094.fe3a2eacea8dd8daae28302db14142ed. 2023-02-11 03:56:13,587 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-02-11 03:56:13,964 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/30759f6081f5ddc390558cd75a0299b8/.tmp/family2/9d98065c542c4813a153663332fc15e4 2023-02-11 03:56:13,968 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/30759f6081f5ddc390558cd75a0299b8/.tmp/family1/394aad0fbd074d8093fb08363fb9a75f as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/30759f6081f5ddc390558cd75a0299b8/family1/394aad0fbd074d8093fb08363fb9a75f 2023-02-11 03:56:13,993 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/30759f6081f5ddc390558cd75a0299b8/family1/394aad0fbd074d8093fb08363fb9a75f, entries=4, sequenceid=5, filesize=5.5 K 2023-02-11 03:56:13,994 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/30759f6081f5ddc390558cd75a0299b8/.tmp/family2/9d98065c542c4813a153663332fc15e4 as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/30759f6081f5ddc390558cd75a0299b8/family2/9d98065c542c4813a153663332fc15e4 2023-02-11 03:56:13,997 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/30759f6081f5ddc390558cd75a0299b8/family2/9d98065c542c4813a153663332fc15e4, entries=2, sequenceid=5, filesize=5.1 K 2023-02-11 03:56:13,998 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-2] regionserver.HRegion(2947): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for 30759f6081f5ddc390558cd75a0299b8 in 1752ms, sequenceid=5, compaction requested=false 2023-02-11 03:56:13,998 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-2] regionserver.HRegion(2445): Flush status journal for 30759f6081f5ddc390558cd75a0299b8: 2023-02-11 03:56:13,998 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,2,1676087770094.30759f6081f5ddc390558cd75a0299b8. 2023-02-11 03:56:13,998 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-02-11 03:56:13,998 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-02-11 03:56:13,998 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:13,998 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionFromAdmin' locally acquired 2023-02-11 03:56:13,998 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37011,1676087618428' joining acquired barrier for procedure (testMajorCompactionFromAdmin) in zk 2023-02-11 03:56:14,004 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:14,004 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-11 03:56:14,004 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:14,004 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:56:14,004 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:56:14,004 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-11 03:56:14,004 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionFromAdmin' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 03:56:14,005 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:56:14,005 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:56:14,005 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-11 03:56:14,005 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:14,005 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:56:14,006 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37011,1676087618428' joining acquired barrier for procedure 'testMajorCompactionFromAdmin' on coordinator 2023-02-11 03:56:14,006 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@42a9a1ad[Count = 0] remaining members to acquire global barrier 2023-02-11 03:56:14,006 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionFromAdmin' starting 'in-barrier' execution. 2023-02-11 03:56:14,006 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-11 03:56:14,007 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-11 03:56:14,007 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-11 03:56:14,007 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-11 03:56:14,007 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionFromAdmin' received 'reached' from coordinator. 2023-02-11 03:56:14,007 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionFromAdmin' locally completed 2023-02-11 03:56:14,007 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionFromAdmin' completed for member 'jenkins-hbase4.apache.org,37011,1676087618428' in zk 2023-02-11 03:56:14,007 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:14,007 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 03:56:14,008 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionFromAdmin' has notified controller of completion 2023-02-11 03:56:14,008 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:56:14,008 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:14,008 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionFromAdmin' completed. 2023-02-11 03:56:14,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:14,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:56:14,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:56:14,009 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:56:14,009 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:56:14,009 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-11 03:56:14,009 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:14,009 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:56:14,010 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-11 03:56:14,010 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:14,010 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionFromAdmin' member 'jenkins-hbase4.apache.org,37011,1676087618428': 2023-02-11 03:56:14,010 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37011,1676087618428' released barrier for procedure'testMajorCompactionFromAdmin', counting down latch. Waiting for 0 more 2023-02-11 03:56:14,010 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionFromAdmin' execution completed 2023-02-11 03:56:14,010 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 03:56:14,010 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 03:56:14,010 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionFromAdmin 2023-02-11 03:56:14,011 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionFromAdminincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 03:56:14,012 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-11 03:56:14,012 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-11 03:56:14,012 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-11 03:56:14,012 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-11 03:56:14,012 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:56:14,012 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:56:14,012 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:56:14,012 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-11 03:56:14,012 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:14,012 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:56:14,012 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:56:14,012 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:56:14,013 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-11 03:56:14,013 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-11 03:56:14,013 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:56:14,013 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:14,013 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-11 03:56:14,013 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:14,014 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:56:14,014 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-11 03:56:14,014 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:14,016 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:14,016 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:56:14,016 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-02-11 03:56:14,016 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39921] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionFromAdmin' 2023-02-11 03:56:14,016 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39921] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 03:56:14,016 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-02-11 03:56:14,016 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:56:14,016 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:56:14,016 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin because node does not exist (not an error) 2023-02-11 03:56:14,016 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:56:14,016 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionFromAdmin'' to complete. (max 300000 ms per retry) 2023-02-11 03:56:14,016 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:56:14,016 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:14,016 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 03:56:14,016 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-11 03:56:14,017 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-11 03:56:14,017 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-11 03:56:14,017 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:56:14,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:56:14,117 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 03:56:14,117 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39921] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 03:56:14,118 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37011] regionserver.HRegion(8801): writing data to region testMajorCompactionFromAdmin,,1676087770094.953c5751d1228a55e688b51c4b8bb6fe. with WAL disabled. Data may be lost in the event of a crash. 2023-02-11 03:56:14,119 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37011] regionserver.HRegion(8801): writing data to region testMajorCompactionFromAdmin,1,1676087770094.fe3a2eacea8dd8daae28302db14142ed. with WAL disabled. Data may be lost in the event of a crash. 2023-02-11 03:56:14,119 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37011] regionserver.HRegion(8801): writing data to region testMajorCompactionFromAdmin,2,1676087770094.30759f6081f5ddc390558cd75a0299b8. with WAL disabled. Data may be lost in the event of a crash. 2023-02-11 03:56:14,121 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39921] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 03:56:14,122 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39921] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionFromAdmin was in running list but was completed. Accepting new attempt. 2023-02-11 03:56:14,122 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39921] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionFromAdmin 2023-02-11 03:56:14,122 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionFromAdmin' 2023-02-11 03:56:14,122 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:56:14,122 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionFromAdmin' starting 'acquire' 2023-02-11 03:56:14,123 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionFromAdmin', kicking off acquire phase on members. 2023-02-11 03:56:14,123 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-11 03:56:14,123 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-02-11 03:56:14,124 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:56:14,124 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:14,124 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:56:14,124 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:56:14,124 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:14,124 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 03:56:14,124 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-02-11 03:56:14,125 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-11 03:56:14,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 03:56:14,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-02-11 03:56:14,125 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionFromAdmin 2023-02-11 03:56:14,125 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionFromAdmin is still present, removing 2023-02-11 03:56:14,125 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionFromAdmin 2023-02-11 03:56:14,125 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionFromAdmin' with timeout 60000ms 2023-02-11 03:56:14,125 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:56:14,125 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionFromAdmin' starting 'acquire' stage 2023-02-11 03:56:14,126 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,1,1676087770094.fe3a2eacea8dd8daae28302db14142ed. 2023-02-11 03:56:14,126 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,,1676087770094.953c5751d1228a55e688b51c4b8bb6fe. 2023-02-11 03:56:14,126 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,2,1676087770094.30759f6081f5ddc390558cd75a0299b8. 2023-02-11 03:56:14,126 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,,1676087770094.953c5751d1228a55e688b51c4b8bb6fe. started... 2023-02-11 03:56:14,126 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,2,1676087770094.30759f6081f5ddc390558cd75a0299b8. started... 2023-02-11 03:56:14,126 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-02-11 03:56:14,126 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 03:56:14,126 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-1] regionserver.HRegion(2744): Flushing 953c5751d1228a55e688b51c4b8bb6fe 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-02-11 03:56:14,126 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-2] regionserver.HRegion(2744): Flushing 30759f6081f5ddc390558cd75a0299b8 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-02-11 03:56:14,126 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,1,1676087770094.fe3a2eacea8dd8daae28302db14142ed. started... 2023-02-11 03:56:14,126 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing fe3a2eacea8dd8daae28302db14142ed 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-02-11 03:56:14,582 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202302117c820746ccc34c1e9dd444cba8a9501d to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c202302117c820746ccc34c1e9dd444cba8a9501d 2023-02-11 03:56:14,584 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230211d4c3f669250f4f51bd4f7e95a3d83f42 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e20230211d4c3f669250f4f51bd4f7e95a3d83f42 2023-02-11 03:56:14,584 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202302116afa6252614047f28eaaf31c5ede4810 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b202302116afa6252614047f28eaaf31c5ede4810 2023-02-11 03:56:14,996 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/30759f6081f5ddc390558cd75a0299b8/.tmp/family1/0c688f0c757149c789c43cea2c343feb 2023-02-11 03:56:15,004 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/fe3a2eacea8dd8daae28302db14142ed/.tmp/family1/98f3723d9f6a461b8d01e0758908e398 2023-02-11 03:56:15,004 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/953c5751d1228a55e688b51c4b8bb6fe/.tmp/family1/fd99ec7b081c47b0b139a5010d997ac4 2023-02-11 03:56:15,034 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230211127db1c7640c419b889ebf039548913e to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c4ca4238a0b923820dcc509a6f75849b20230211127db1c7640c419b889ebf039548913e 2023-02-11 03:56:15,430 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202302116d6a906536a04db888fcb61d689a9e4e to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c81e728d9d4c2f636f067f89cc14862c202302116d6a906536a04db888fcb61d689a9e4e 2023-02-11 03:56:15,446 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/fe3a2eacea8dd8daae28302db14142ed/.tmp/family2/9ac549495c004b308436f6f6d1e539d7 2023-02-11 03:56:15,449 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/fe3a2eacea8dd8daae28302db14142ed/.tmp/family1/98f3723d9f6a461b8d01e0758908e398 as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/fe3a2eacea8dd8daae28302db14142ed/family1/98f3723d9f6a461b8d01e0758908e398 2023-02-11 03:56:15,452 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2023021161951c7086814161aa44c39d82230b44 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/d41d8cd98f00b204e9800998ecf8427e2023021161951c7086814161aa44c39d82230b44 2023-02-11 03:56:15,472 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/fe3a2eacea8dd8daae28302db14142ed/family1/98f3723d9f6a461b8d01e0758908e398, entries=4, sequenceid=9, filesize=5.5 K 2023-02-11 03:56:15,473 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/fe3a2eacea8dd8daae28302db14142ed/.tmp/family2/9ac549495c004b308436f6f6d1e539d7 as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/fe3a2eacea8dd8daae28302db14142ed/family2/9ac549495c004b308436f6f6d1e539d7 2023-02-11 03:56:15,477 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/fe3a2eacea8dd8daae28302db14142ed/family2/9ac549495c004b308436f6f6d1e539d7, entries=2, sequenceid=9, filesize=5.1 K 2023-02-11 03:56:15,478 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for fe3a2eacea8dd8daae28302db14142ed in 1352ms, sequenceid=9, compaction requested=false 2023-02-11 03:56:15,478 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for fe3a2eacea8dd8daae28302db14142ed: 2023-02-11 03:56:15,478 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,1,1676087770094.fe3a2eacea8dd8daae28302db14142ed. 2023-02-11 03:56:15,478 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-02-11 03:56:15,835 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/30759f6081f5ddc390558cd75a0299b8/.tmp/family2/72fa679c42fa42ed913c4140d456aeed 2023-02-11 03:56:15,839 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/30759f6081f5ddc390558cd75a0299b8/.tmp/family1/0c688f0c757149c789c43cea2c343feb as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/30759f6081f5ddc390558cd75a0299b8/family1/0c688f0c757149c789c43cea2c343feb 2023-02-11 03:56:15,861 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/30759f6081f5ddc390558cd75a0299b8/family1/0c688f0c757149c789c43cea2c343feb, entries=4, sequenceid=9, filesize=5.5 K 2023-02-11 03:56:15,862 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/953c5751d1228a55e688b51c4b8bb6fe/.tmp/family2/688326961d3a40008a354222d19ec75c 2023-02-11 03:56:15,862 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/30759f6081f5ddc390558cd75a0299b8/.tmp/family2/72fa679c42fa42ed913c4140d456aeed as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/30759f6081f5ddc390558cd75a0299b8/family2/72fa679c42fa42ed913c4140d456aeed 2023-02-11 03:56:15,865 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/30759f6081f5ddc390558cd75a0299b8/family2/72fa679c42fa42ed913c4140d456aeed, entries=2, sequenceid=9, filesize=5.1 K 2023-02-11 03:56:15,865 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/953c5751d1228a55e688b51c4b8bb6fe/.tmp/family1/fd99ec7b081c47b0b139a5010d997ac4 as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/953c5751d1228a55e688b51c4b8bb6fe/family1/fd99ec7b081c47b0b139a5010d997ac4 2023-02-11 03:56:15,866 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-2] regionserver.HRegion(2947): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for 30759f6081f5ddc390558cd75a0299b8 in 1740ms, sequenceid=9, compaction requested=false 2023-02-11 03:56:15,866 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-2] regionserver.HRegion(2445): Flush status journal for 30759f6081f5ddc390558cd75a0299b8: 2023-02-11 03:56:15,866 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,2,1676087770094.30759f6081f5ddc390558cd75a0299b8. 2023-02-11 03:56:15,866 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-02-11 03:56:15,887 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/953c5751d1228a55e688b51c4b8bb6fe/family1/fd99ec7b081c47b0b139a5010d997ac4, entries=4, sequenceid=9, filesize=5.5 K 2023-02-11 03:56:15,887 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/953c5751d1228a55e688b51c4b8bb6fe/.tmp/family2/688326961d3a40008a354222d19ec75c as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/953c5751d1228a55e688b51c4b8bb6fe/family2/688326961d3a40008a354222d19ec75c 2023-02-11 03:56:15,891 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/953c5751d1228a55e688b51c4b8bb6fe/family2/688326961d3a40008a354222d19ec75c, entries=2, sequenceid=9, filesize=5.1 K 2023-02-11 03:56:15,891 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-1] regionserver.HRegion(2947): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for 953c5751d1228a55e688b51c4b8bb6fe in 1765ms, sequenceid=9, compaction requested=false 2023-02-11 03:56:15,891 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-1] regionserver.HRegion(2445): Flush status journal for 953c5751d1228a55e688b51c4b8bb6fe: 2023-02-11 03:56:15,891 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,,1676087770094.953c5751d1228a55e688b51c4b8bb6fe. 2023-02-11 03:56:15,891 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-02-11 03:56:15,891 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-02-11 03:56:15,891 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:15,891 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionFromAdmin' locally acquired 2023-02-11 03:56:15,892 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37011,1676087618428' joining acquired barrier for procedure (testMajorCompactionFromAdmin) in zk 2023-02-11 03:56:15,893 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:15,893 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-11 03:56:15,893 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:15,893 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:56:15,893 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:56:15,893 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-11 03:56:15,893 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionFromAdmin' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 03:56:15,893 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:56:15,894 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:56:15,894 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-11 03:56:15,894 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:15,894 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:56:15,895 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37011,1676087618428' joining acquired barrier for procedure 'testMajorCompactionFromAdmin' on coordinator 2023-02-11 03:56:15,895 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@216a4701[Count = 0] remaining members to acquire global barrier 2023-02-11 03:56:15,895 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionFromAdmin' starting 'in-barrier' execution. 2023-02-11 03:56:15,895 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-11 03:56:15,896 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-11 03:56:15,896 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-11 03:56:15,896 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-11 03:56:15,896 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionFromAdmin' received 'reached' from coordinator. 2023-02-11 03:56:15,896 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionFromAdmin' locally completed 2023-02-11 03:56:15,896 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionFromAdmin' completed for member 'jenkins-hbase4.apache.org,37011,1676087618428' in zk 2023-02-11 03:56:15,896 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:15,896 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 03:56:15,898 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionFromAdmin' has notified controller of completion 2023-02-11 03:56:15,898 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:15,898 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:56:15,898 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionFromAdmin' completed. 2023-02-11 03:56:15,898 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:15,898 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:56:15,898 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:56:15,899 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:56:15,899 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:56:15,899 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-11 03:56:15,899 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:15,900 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:56:15,900 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-11 03:56:15,900 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:15,900 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionFromAdmin' member 'jenkins-hbase4.apache.org,37011,1676087618428': 2023-02-11 03:56:15,900 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37011,1676087618428' released barrier for procedure'testMajorCompactionFromAdmin', counting down latch. Waiting for 0 more 2023-02-11 03:56:15,900 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionFromAdmin' execution completed 2023-02-11 03:56:15,901 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 03:56:15,901 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 03:56:15,901 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionFromAdmin 2023-02-11 03:56:15,901 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionFromAdminincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 03:56:15,902 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-11 03:56:15,902 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-11 03:56:15,902 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:56:15,902 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-11 03:56:15,902 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-11 03:56:15,902 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-11 03:56:15,902 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:56:15,902 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:56:15,902 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:56:15,902 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:56:15,903 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-11 03:56:15,903 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:56:15,903 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:15,903 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-11 03:56:15,903 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:56:15,903 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-11 03:56:15,904 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:15,904 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:15,904 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:56:15,904 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-11 03:56:15,905 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:15,906 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:15,906 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:56:15,906 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-02-11 03:56:15,906 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:56:15,906 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-02-11 03:56:15,906 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:56:15,906 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:56:15,906 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:15,906 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin because node does not exist (not an error) 2023-02-11 03:56:15,906 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:56:15,906 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39921] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionFromAdmin' 2023-02-11 03:56:15,906 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39921] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 03:56:15,906 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-11 03:56:15,906 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-11 03:56:15,906 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-11 03:56:15,906 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:56:15,906 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionFromAdmin'' to complete. (max 300000 ms per retry) 2023-02-11 03:56:15,906 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 03:56:15,906 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:56:16,006 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 03:56:16,007 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39921] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 03:56:16,008 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37011] regionserver.HRegion(8801): writing data to region testMajorCompactionFromAdmin,,1676087770094.953c5751d1228a55e688b51c4b8bb6fe. with WAL disabled. Data may be lost in the event of a crash. 2023-02-11 03:56:16,008 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37011] regionserver.HRegion(8801): writing data to region testMajorCompactionFromAdmin,1,1676087770094.fe3a2eacea8dd8daae28302db14142ed. with WAL disabled. Data may be lost in the event of a crash. 2023-02-11 03:56:16,009 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37011] regionserver.HRegion(8801): writing data to region testMajorCompactionFromAdmin,2,1676087770094.30759f6081f5ddc390558cd75a0299b8. with WAL disabled. Data may be lost in the event of a crash. 2023-02-11 03:56:16,011 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39921] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 03:56:16,012 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39921] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionFromAdmin was in running list but was completed. Accepting new attempt. 2023-02-11 03:56:16,012 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39921] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionFromAdmin 2023-02-11 03:56:16,012 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionFromAdmin' 2023-02-11 03:56:16,012 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:56:16,012 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionFromAdmin' starting 'acquire' 2023-02-11 03:56:16,012 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionFromAdmin', kicking off acquire phase on members. 2023-02-11 03:56:16,013 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-11 03:56:16,013 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-02-11 03:56:16,014 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:16,014 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:56:16,014 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:56:16,014 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:56:16,014 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:16,014 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 03:56:16,014 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-02-11 03:56:16,015 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-11 03:56:16,015 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 03:56:16,015 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-02-11 03:56:16,015 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionFromAdmin 2023-02-11 03:56:16,015 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionFromAdmin is still present, removing 2023-02-11 03:56:16,015 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionFromAdmin 2023-02-11 03:56:16,015 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionFromAdmin' with timeout 60000ms 2023-02-11 03:56:16,015 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:56:16,015 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionFromAdmin' starting 'acquire' stage 2023-02-11 03:56:16,015 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,1,1676087770094.fe3a2eacea8dd8daae28302db14142ed. 2023-02-11 03:56:16,015 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,1,1676087770094.fe3a2eacea8dd8daae28302db14142ed. started... 2023-02-11 03:56:16,015 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,,1676087770094.953c5751d1228a55e688b51c4b8bb6fe. 2023-02-11 03:56:16,016 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-02-11 03:56:16,016 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,2,1676087770094.30759f6081f5ddc390558cd75a0299b8. 2023-02-11 03:56:16,016 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 03:56:16,016 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing fe3a2eacea8dd8daae28302db14142ed 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-02-11 03:56:16,016 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,,1676087770094.953c5751d1228a55e688b51c4b8bb6fe. started... 2023-02-11 03:56:16,016 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,2,1676087770094.30759f6081f5ddc390558cd75a0299b8. started... 2023-02-11 03:56:16,016 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-1] regionserver.HRegion(2744): Flushing 953c5751d1228a55e688b51c4b8bb6fe 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-02-11 03:56:16,016 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-2] regionserver.HRegion(2744): Flushing 30759f6081f5ddc390558cd75a0299b8 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-02-11 03:56:16,075 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202302116d9da738ee2947068908fb14fc71680c to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b202302116d9da738ee2947068908fb14fc71680c 2023-02-11 03:56:16,084 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230211d61b27642992406b913fb12b434f7a3e to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e20230211d61b27642992406b913fb12b434f7a3e 2023-02-11 03:56:16,470 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c2023021141c897fe62da403797b0c35b4deb7fda to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c2023021141c897fe62da403797b0c35b4deb7fda 2023-02-11 03:56:16,487 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/30759f6081f5ddc390558cd75a0299b8/.tmp/family1/bb80ff603e2e49ddaba5457a7cd04a14 2023-02-11 03:56:16,495 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/fe3a2eacea8dd8daae28302db14142ed/.tmp/family1/a70037e87d3345589853ba1104a091c7 2023-02-11 03:56:16,510 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/953c5751d1228a55e688b51c4b8bb6fe/.tmp/family1/f763e1f9e5484f0483d251fc2b86b0e7 2023-02-11 03:56:16,550 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c2023021163099cde0938407abee399a29a909f34 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c81e728d9d4c2f636f067f89cc14862c2023021163099cde0938407abee399a29a909f34 2023-02-11 03:56:16,564 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/30759f6081f5ddc390558cd75a0299b8/.tmp/family2/0b1c2c1de15a415488c64db1e9c2a924 2023-02-11 03:56:16,568 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/30759f6081f5ddc390558cd75a0299b8/.tmp/family1/bb80ff603e2e49ddaba5457a7cd04a14 as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/30759f6081f5ddc390558cd75a0299b8/family1/bb80ff603e2e49ddaba5457a7cd04a14 2023-02-11 03:56:16,591 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/30759f6081f5ddc390558cd75a0299b8/family1/bb80ff603e2e49ddaba5457a7cd04a14, entries=4, sequenceid=13, filesize=5.5 K 2023-02-11 03:56:16,592 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/30759f6081f5ddc390558cd75a0299b8/.tmp/family2/0b1c2c1de15a415488c64db1e9c2a924 as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/30759f6081f5ddc390558cd75a0299b8/family2/0b1c2c1de15a415488c64db1e9c2a924 2023-02-11 03:56:16,595 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/30759f6081f5ddc390558cd75a0299b8/family2/0b1c2c1de15a415488c64db1e9c2a924, entries=2, sequenceid=13, filesize=5.1 K 2023-02-11 03:56:16,595 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-2] regionserver.HRegion(2947): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for 30759f6081f5ddc390558cd75a0299b8 in 579ms, sequenceid=13, compaction requested=true 2023-02-11 03:56:16,596 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-2] regionserver.HRegion(2445): Flush status journal for 30759f6081f5ddc390558cd75a0299b8: 2023-02-11 03:56:16,596 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,2,1676087770094.30759f6081f5ddc390558cd75a0299b8. 2023-02-11 03:56:16,596 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-02-11 03:56:16,810 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-11 03:56:16,949 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230211711f43c8c2e34418a80ba90f0293cbf3 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c4ca4238a0b923820dcc509a6f75849b20230211711f43c8c2e34418a80ba90f0293cbf3 2023-02-11 03:56:16,960 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230211c6779fdd4f3f4f2490d8fe041d2358b9 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/d41d8cd98f00b204e9800998ecf8427e20230211c6779fdd4f3f4f2490d8fe041d2358b9 2023-02-11 03:56:17,354 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/fe3a2eacea8dd8daae28302db14142ed/.tmp/family2/4341eff9ec7443839ebc21d9e8f0d8a4 2023-02-11 03:56:17,357 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/fe3a2eacea8dd8daae28302db14142ed/.tmp/family1/a70037e87d3345589853ba1104a091c7 as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/fe3a2eacea8dd8daae28302db14142ed/family1/a70037e87d3345589853ba1104a091c7 2023-02-11 03:56:17,365 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/953c5751d1228a55e688b51c4b8bb6fe/.tmp/family2/548fe02449c644c49b4e45e3c789ec3e 2023-02-11 03:56:17,368 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/953c5751d1228a55e688b51c4b8bb6fe/.tmp/family1/f763e1f9e5484f0483d251fc2b86b0e7 as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/953c5751d1228a55e688b51c4b8bb6fe/family1/f763e1f9e5484f0483d251fc2b86b0e7 2023-02-11 03:56:17,381 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/fe3a2eacea8dd8daae28302db14142ed/family1/a70037e87d3345589853ba1104a091c7, entries=4, sequenceid=13, filesize=5.5 K 2023-02-11 03:56:17,381 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/fe3a2eacea8dd8daae28302db14142ed/.tmp/family2/4341eff9ec7443839ebc21d9e8f0d8a4 as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/fe3a2eacea8dd8daae28302db14142ed/family2/4341eff9ec7443839ebc21d9e8f0d8a4 2023-02-11 03:56:17,385 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/fe3a2eacea8dd8daae28302db14142ed/family2/4341eff9ec7443839ebc21d9e8f0d8a4, entries=2, sequenceid=13, filesize=5.1 K 2023-02-11 03:56:17,386 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for fe3a2eacea8dd8daae28302db14142ed in 1370ms, sequenceid=13, compaction requested=true 2023-02-11 03:56:17,386 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for fe3a2eacea8dd8daae28302db14142ed: 2023-02-11 03:56:17,386 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,1,1676087770094.fe3a2eacea8dd8daae28302db14142ed. 2023-02-11 03:56:17,386 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-02-11 03:56:17,391 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/953c5751d1228a55e688b51c4b8bb6fe/family1/f763e1f9e5484f0483d251fc2b86b0e7, entries=4, sequenceid=13, filesize=5.5 K 2023-02-11 03:56:17,391 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/953c5751d1228a55e688b51c4b8bb6fe/.tmp/family2/548fe02449c644c49b4e45e3c789ec3e as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/953c5751d1228a55e688b51c4b8bb6fe/family2/548fe02449c644c49b4e45e3c789ec3e 2023-02-11 03:56:17,395 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/953c5751d1228a55e688b51c4b8bb6fe/family2/548fe02449c644c49b4e45e3c789ec3e, entries=2, sequenceid=13, filesize=5.1 K 2023-02-11 03:56:17,396 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-1] regionserver.HRegion(2947): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for 953c5751d1228a55e688b51c4b8bb6fe in 1379ms, sequenceid=13, compaction requested=true 2023-02-11 03:56:17,396 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-1] regionserver.HRegion(2445): Flush status journal for 953c5751d1228a55e688b51c4b8bb6fe: 2023-02-11 03:56:17,396 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,,1676087770094.953c5751d1228a55e688b51c4b8bb6fe. 2023-02-11 03:56:17,396 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-02-11 03:56:17,396 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-02-11 03:56:17,396 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:17,396 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionFromAdmin' locally acquired 2023-02-11 03:56:17,396 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37011,1676087618428' joining acquired barrier for procedure (testMajorCompactionFromAdmin) in zk 2023-02-11 03:56:17,397 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:17,397 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-11 03:56:17,397 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:17,397 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:56:17,397 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:56:17,398 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-11 03:56:17,398 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionFromAdmin' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 03:56:17,398 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:56:17,398 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:56:17,398 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-11 03:56:17,398 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:17,399 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:56:17,399 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37011,1676087618428' joining acquired barrier for procedure 'testMajorCompactionFromAdmin' on coordinator 2023-02-11 03:56:17,399 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4ff5c668[Count = 0] remaining members to acquire global barrier 2023-02-11 03:56:17,399 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionFromAdmin' starting 'in-barrier' execution. 2023-02-11 03:56:17,399 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-11 03:56:17,401 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-11 03:56:17,401 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-11 03:56:17,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-11 03:56:17,401 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionFromAdmin' received 'reached' from coordinator. 2023-02-11 03:56:17,401 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionFromAdmin' locally completed 2023-02-11 03:56:17,401 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:17,402 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 03:56:17,401 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionFromAdmin' completed for member 'jenkins-hbase4.apache.org,37011,1676087618428' in zk 2023-02-11 03:56:17,403 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionFromAdmin' has notified controller of completion 2023-02-11 03:56:17,403 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:17,403 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:56:17,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:17,403 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionFromAdmin' completed. 2023-02-11 03:56:17,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:56:17,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:56:17,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:56:17,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:56:17,404 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-11 03:56:17,404 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:17,404 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:56:17,404 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-11 03:56:17,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:17,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionFromAdmin' member 'jenkins-hbase4.apache.org,37011,1676087618428': 2023-02-11 03:56:17,405 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37011,1676087618428' released barrier for procedure'testMajorCompactionFromAdmin', counting down latch. Waiting for 0 more 2023-02-11 03:56:17,405 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionFromAdmin' execution completed 2023-02-11 03:56:17,405 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 03:56:17,405 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 03:56:17,405 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionFromAdmin 2023-02-11 03:56:17,405 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionFromAdminincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 03:56:17,407 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-11 03:56:17,407 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-11 03:56:17,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-11 03:56:17,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:56:17,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:56:17,407 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-11 03:56:17,407 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:56:17,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-11 03:56:17,407 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:56:17,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:56:17,407 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:17,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:56:17,408 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-11 03:56:17,408 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-11 03:56:17,408 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:56:17,409 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-11 03:56:17,409 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:17,409 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:17,409 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:56:17,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-11 03:56:17,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:17,411 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:17,411 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:56:17,411 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:56:17,411 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:56:17,411 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:56:17,411 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39921] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionFromAdmin' 2023-02-11 03:56:17,411 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39921] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 03:56:17,411 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin because node does not exist (not an error) 2023-02-11 03:56:17,411 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-02-11 03:56:17,411 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:56:17,411 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-02-11 03:56:17,411 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:17,412 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionFromAdmin'' to complete. (max 300000 ms per retry) 2023-02-11 03:56:17,412 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-11 03:56:17,412 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:56:17,412 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:56:17,412 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-11 03:56:17,412 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 03:56:17,412 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-11 03:56:17,512 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 03:56:17,513 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39921] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 03:56:17,514 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37011] regionserver.HRegion(8801): writing data to region testMajorCompactionFromAdmin,,1676087770094.953c5751d1228a55e688b51c4b8bb6fe. with WAL disabled. Data may be lost in the event of a crash. 2023-02-11 03:56:17,514 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37011] regionserver.HRegion(8801): writing data to region testMajorCompactionFromAdmin,1,1676087770094.fe3a2eacea8dd8daae28302db14142ed. with WAL disabled. Data may be lost in the event of a crash. 2023-02-11 03:56:17,514 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37011] regionserver.HRegion(8801): writing data to region testMajorCompactionFromAdmin,2,1676087770094.30759f6081f5ddc390558cd75a0299b8. with WAL disabled. Data may be lost in the event of a crash. 2023-02-11 03:56:17,516 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39921] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 03:56:17,518 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39921] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionFromAdmin was in running list but was completed. Accepting new attempt. 2023-02-11 03:56:17,518 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39921] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionFromAdmin 2023-02-11 03:56:17,518 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionFromAdmin' 2023-02-11 03:56:17,518 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:56:17,518 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionFromAdmin' starting 'acquire' 2023-02-11 03:56:17,518 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionFromAdmin', kicking off acquire phase on members. 2023-02-11 03:56:17,518 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-11 03:56:17,518 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-02-11 03:56:17,522 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:56:17,522 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:17,523 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:56:17,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:56:17,523 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:17,523 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 03:56:17,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-02-11 03:56:17,523 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-11 03:56:17,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 03:56:17,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-02-11 03:56:17,524 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionFromAdmin 2023-02-11 03:56:17,524 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionFromAdmin is still present, removing 2023-02-11 03:56:17,524 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionFromAdmin 2023-02-11 03:56:17,524 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionFromAdmin' with timeout 60000ms 2023-02-11 03:56:17,524 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:56:17,524 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionFromAdmin' starting 'acquire' stage 2023-02-11 03:56:17,524 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,1,1676087770094.fe3a2eacea8dd8daae28302db14142ed. 2023-02-11 03:56:17,524 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,1,1676087770094.fe3a2eacea8dd8daae28302db14142ed. started... 2023-02-11 03:56:17,524 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-02-11 03:56:17,524 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 03:56:17,524 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,,1676087770094.953c5751d1228a55e688b51c4b8bb6fe. 2023-02-11 03:56:17,524 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,2,1676087770094.30759f6081f5ddc390558cd75a0299b8. 2023-02-11 03:56:17,524 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,,1676087770094.953c5751d1228a55e688b51c4b8bb6fe. started... 2023-02-11 03:56:17,524 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,2,1676087770094.30759f6081f5ddc390558cd75a0299b8. started... 2023-02-11 03:56:17,524 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing fe3a2eacea8dd8daae28302db14142ed 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-02-11 03:56:17,524 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-1] regionserver.HRegion(2744): Flushing 953c5751d1228a55e688b51c4b8bb6fe 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-02-11 03:56:17,525 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-2] regionserver.HRegion(2744): Flushing 30759f6081f5ddc390558cd75a0299b8 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-02-11 03:56:17,993 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-02-11 03:56:17,993 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x5282e94f to 127.0.0.1:53383 2023-02-11 03:56:17,993 DEBUG [Finalizer] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-11 03:56:17,993 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-02-11 03:56:17,994 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x187963cb to 127.0.0.1:53383 2023-02-11 03:56:17,994 DEBUG [Finalizer] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-11 03:56:17,996 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230211ef0836e42e8e4aed9e6240d5f7dc8b91 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20230211ef0836e42e8e4aed9e6240d5f7dc8b91 2023-02-11 03:56:17,997 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230211b07eb6f6c0d746748f2f3d5cad956402 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b20230211b07eb6f6c0d746748f2f3d5cad956402 2023-02-11 03:56:18,003 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230211ebe88bc7cbf04c0ca3a4f84333645941 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e20230211ebe88bc7cbf04c0ca3a4f84333645941 2023-02-11 03:56:18,413 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/fe3a2eacea8dd8daae28302db14142ed/.tmp/family1/90203d6e75a841a6a5d172b0b422cd9e 2023-02-11 03:56:18,418 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/953c5751d1228a55e688b51c4b8bb6fe/.tmp/family1/09fb95e4f1e74b45b68b3aacca080f46 2023-02-11 03:56:18,418 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/30759f6081f5ddc390558cd75a0299b8/.tmp/family1/a27b5488126c4e97847bea80eeb14cea 2023-02-11 03:56:18,456 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202302111a6eb6942316466c8419e733d7a7603e to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c4ca4238a0b923820dcc509a6f75849b202302111a6eb6942316466c8419e733d7a7603e 2023-02-11 03:56:18,464 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/fe3a2eacea8dd8daae28302db14142ed/.tmp/family2/a1de535155ef4b41b92da28d66e43a64 2023-02-11 03:56:18,468 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230211c0e91f1797344d729c5163c4e47b3e7b to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c81e728d9d4c2f636f067f89cc14862c20230211c0e91f1797344d729c5163c4e47b3e7b 2023-02-11 03:56:18,469 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/fe3a2eacea8dd8daae28302db14142ed/.tmp/family1/90203d6e75a841a6a5d172b0b422cd9e as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/fe3a2eacea8dd8daae28302db14142ed/family1/90203d6e75a841a6a5d172b0b422cd9e 2023-02-11 03:56:18,479 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/30759f6081f5ddc390558cd75a0299b8/.tmp/family2/714b21396662486ea76d3db63c3145ea 2023-02-11 03:56:18,482 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/30759f6081f5ddc390558cd75a0299b8/.tmp/family1/a27b5488126c4e97847bea80eeb14cea as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/30759f6081f5ddc390558cd75a0299b8/family1/a27b5488126c4e97847bea80eeb14cea 2023-02-11 03:56:18,505 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/30759f6081f5ddc390558cd75a0299b8/family1/a27b5488126c4e97847bea80eeb14cea, entries=4, sequenceid=17, filesize=5.5 K 2023-02-11 03:56:18,505 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/fe3a2eacea8dd8daae28302db14142ed/family1/90203d6e75a841a6a5d172b0b422cd9e, entries=4, sequenceid=17, filesize=5.5 K 2023-02-11 03:56:18,505 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/30759f6081f5ddc390558cd75a0299b8/.tmp/family2/714b21396662486ea76d3db63c3145ea as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/30759f6081f5ddc390558cd75a0299b8/family2/714b21396662486ea76d3db63c3145ea 2023-02-11 03:56:18,505 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/fe3a2eacea8dd8daae28302db14142ed/.tmp/family2/a1de535155ef4b41b92da28d66e43a64 as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/fe3a2eacea8dd8daae28302db14142ed/family2/a1de535155ef4b41b92da28d66e43a64 2023-02-11 03:56:18,509 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/fe3a2eacea8dd8daae28302db14142ed/family2/a1de535155ef4b41b92da28d66e43a64, entries=2, sequenceid=17, filesize=5.1 K 2023-02-11 03:56:18,510 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/30759f6081f5ddc390558cd75a0299b8/family2/714b21396662486ea76d3db63c3145ea, entries=2, sequenceid=17, filesize=5.1 K 2023-02-11 03:56:18,510 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for fe3a2eacea8dd8daae28302db14142ed in 986ms, sequenceid=17, compaction requested=true 2023-02-11 03:56:18,510 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-2] regionserver.HRegion(2947): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for 30759f6081f5ddc390558cd75a0299b8 in 986ms, sequenceid=17, compaction requested=true 2023-02-11 03:56:18,510 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for fe3a2eacea8dd8daae28302db14142ed: 2023-02-11 03:56:18,510 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-2] regionserver.HRegion(2445): Flush status journal for 30759f6081f5ddc390558cd75a0299b8: 2023-02-11 03:56:18,510 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,1,1676087770094.fe3a2eacea8dd8daae28302db14142ed. 2023-02-11 03:56:18,510 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,2,1676087770094.30759f6081f5ddc390558cd75a0299b8. 2023-02-11 03:56:18,510 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-02-11 03:56:18,511 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-02-11 03:56:18,867 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2023021130867978b01249efa12a8d3ab5b81543 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/d41d8cd98f00b204e9800998ecf8427e2023021130867978b01249efa12a8d3ab5b81543 2023-02-11 03:56:19,272 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/953c5751d1228a55e688b51c4b8bb6fe/.tmp/family2/df4b77f4363d4fe3b8d60b17ef71d390 2023-02-11 03:56:19,276 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/953c5751d1228a55e688b51c4b8bb6fe/.tmp/family1/09fb95e4f1e74b45b68b3aacca080f46 as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/953c5751d1228a55e688b51c4b8bb6fe/family1/09fb95e4f1e74b45b68b3aacca080f46 2023-02-11 03:56:19,299 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/953c5751d1228a55e688b51c4b8bb6fe/family1/09fb95e4f1e74b45b68b3aacca080f46, entries=4, sequenceid=17, filesize=5.5 K 2023-02-11 03:56:19,299 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/953c5751d1228a55e688b51c4b8bb6fe/.tmp/family2/df4b77f4363d4fe3b8d60b17ef71d390 as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/953c5751d1228a55e688b51c4b8bb6fe/family2/df4b77f4363d4fe3b8d60b17ef71d390 2023-02-11 03:56:19,303 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/953c5751d1228a55e688b51c4b8bb6fe/family2/df4b77f4363d4fe3b8d60b17ef71d390, entries=2, sequenceid=17, filesize=5.1 K 2023-02-11 03:56:19,304 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-1] regionserver.HRegion(2947): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for 953c5751d1228a55e688b51c4b8bb6fe in 1779ms, sequenceid=17, compaction requested=true 2023-02-11 03:56:19,304 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-1] regionserver.HRegion(2445): Flush status journal for 953c5751d1228a55e688b51c4b8bb6fe: 2023-02-11 03:56:19,304 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,,1676087770094.953c5751d1228a55e688b51c4b8bb6fe. 2023-02-11 03:56:19,304 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-02-11 03:56:19,304 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-02-11 03:56:19,304 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:19,304 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionFromAdmin' locally acquired 2023-02-11 03:56:19,304 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37011,1676087618428' joining acquired barrier for procedure (testMajorCompactionFromAdmin) in zk 2023-02-11 03:56:19,305 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:19,305 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-11 03:56:19,305 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:19,305 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:56:19,305 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:56:19,306 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-11 03:56:19,306 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionFromAdmin' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 03:56:19,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:56:19,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:56:19,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-11 03:56:19,307 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:19,307 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:56:19,307 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37011,1676087618428' joining acquired barrier for procedure 'testMajorCompactionFromAdmin' on coordinator 2023-02-11 03:56:19,307 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6fc8cc9c[Count = 0] remaining members to acquire global barrier 2023-02-11 03:56:19,307 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionFromAdmin' starting 'in-barrier' execution. 2023-02-11 03:56:19,307 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-11 03:56:19,310 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-11 03:56:19,310 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-11 03:56:19,310 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-11 03:56:19,310 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionFromAdmin' received 'reached' from coordinator. 2023-02-11 03:56:19,310 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionFromAdmin' locally completed 2023-02-11 03:56:19,310 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:19,310 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 03:56:19,310 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionFromAdmin' completed for member 'jenkins-hbase4.apache.org,37011,1676087618428' in zk 2023-02-11 03:56:19,311 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:19,311 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionFromAdmin' has notified controller of completion 2023-02-11 03:56:19,311 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:19,311 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:56:19,311 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:56:19,311 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:56:19,311 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionFromAdmin' completed. 2023-02-11 03:56:19,311 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:56:19,312 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:56:19,312 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-11 03:56:19,312 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:19,312 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:56:19,313 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-11 03:56:19,313 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:19,313 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionFromAdmin' member 'jenkins-hbase4.apache.org,37011,1676087618428': 2023-02-11 03:56:19,313 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37011,1676087618428' released barrier for procedure'testMajorCompactionFromAdmin', counting down latch. Waiting for 0 more 2023-02-11 03:56:19,313 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionFromAdmin' execution completed 2023-02-11 03:56:19,313 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 03:56:19,313 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 03:56:19,313 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionFromAdmin 2023-02-11 03:56:19,313 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionFromAdminincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 03:56:19,314 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-11 03:56:19,314 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-11 03:56:19,314 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-11 03:56:19,315 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:56:19,315 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:56:19,315 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-11 03:56:19,315 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-11 03:56:19,315 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:56:19,315 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:56:19,315 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:19,315 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:56:19,315 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:56:19,315 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-11 03:56:19,315 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-11 03:56:19,315 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:56:19,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-11 03:56:19,316 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:19,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:19,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:56:19,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-11 03:56:19,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:19,318 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:19,318 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:56:19,318 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-02-11 03:56:19,318 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:56:19,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:56:19,318 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:56:19,318 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39921] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionFromAdmin' 2023-02-11 03:56:19,318 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39921] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 03:56:19,318 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:56:19,318 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-02-11 03:56:19,318 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:19,318 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin because node does not exist (not an error) 2023-02-11 03:56:19,318 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-11 03:56:19,318 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-11 03:56:19,318 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-11 03:56:19,318 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionFromAdmin'' to complete. (max 300000 ms per retry) 2023-02-11 03:56:19,318 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 03:56:19,318 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:56:19,319 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:56:19,418 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 03:56:19,419 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39921] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 03:56:19,427 DEBUG [Listener at localhost/46205] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-02-11 03:56:19,433 DEBUG [Listener at localhost/46205] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-02-11 03:56:19,439 DEBUG [Listener at localhost/46205] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-02-11 03:56:19,440 DEBUG [Listener at localhost/46205] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-02-11 03:56:19,518 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39921] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-11 03:56:19,520 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39921] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionFromAdmin was in running list but was completed. Accepting new attempt. 2023-02-11 03:56:19,520 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39921] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionFromAdmin 2023-02-11 03:56:19,520 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionFromAdmin' 2023-02-11 03:56:19,520 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:56:19,520 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionFromAdmin' starting 'acquire' 2023-02-11 03:56:19,520 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionFromAdmin', kicking off acquire phase on members. 2023-02-11 03:56:19,520 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-11 03:56:19,520 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-02-11 03:56:19,523 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:19,523 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:56:19,523 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:56:19,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:56:19,523 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:19,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-02-11 03:56:19,523 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-11 03:56:19,523 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-11 03:56:19,524 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-11 03:56:19,524 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-02-11 03:56:19,524 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionFromAdmin 2023-02-11 03:56:19,524 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionFromAdmin is still present, removing 2023-02-11 03:56:19,524 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionFromAdmin 2023-02-11 03:56:19,524 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionFromAdmin' with timeout 60000ms 2023-02-11 03:56:19,524 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-11 03:56:19,524 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionFromAdmin' starting 'acquire' stage 2023-02-11 03:56:19,524 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,1,1676087770094.fe3a2eacea8dd8daae28302db14142ed. 2023-02-11 03:56:19,524 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-02-11 03:56:19,524 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,,1676087770094.953c5751d1228a55e688b51c4b8bb6fe. 2023-02-11 03:56:19,524 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,2,1676087770094.30759f6081f5ddc390558cd75a0299b8. 2023-02-11 03:56:19,524 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-11 03:56:19,524 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,2,1676087770094.30759f6081f5ddc390558cd75a0299b8. started... 2023-02-11 03:56:19,524 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,1,1676087770094.fe3a2eacea8dd8daae28302db14142ed. started... 2023-02-11 03:56:19,524 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,,1676087770094.953c5751d1228a55e688b51c4b8bb6fe. started... 2023-02-11 03:56:19,525 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-2] regionserver.HRegion(2744): Flushing 30759f6081f5ddc390558cd75a0299b8 2/2 column families, dataSize=138 B heapSize=944 B 2023-02-11 03:56:19,525 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing fe3a2eacea8dd8daae28302db14142ed 2/2 column families, dataSize=138 B heapSize=944 B 2023-02-11 03:56:19,525 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-1] regionserver.HRegion(2744): Flushing 953c5751d1228a55e688b51c4b8bb6fe 2/2 column families, dataSize=138 B heapSize=944 B 2023-02-11 03:56:20,369 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/953c5751d1228a55e688b51c4b8bb6fe/.tmp/family1/77a255e6f2b54178b78a4d3c75f27388 2023-02-11 03:56:20,372 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/fe3a2eacea8dd8daae28302db14142ed/.tmp/family1/249226ed0d3449ba89686cf1c9713498 2023-02-11 03:56:20,372 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/30759f6081f5ddc390558cd75a0299b8/.tmp/family1/dc061f98c53d46a6bee0d2161fc0b0e1 2023-02-11 03:56:20,399 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 77a255e6f2b54178b78a4d3c75f27388 2023-02-11 03:56:20,404 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 249226ed0d3449ba89686cf1c9713498 2023-02-11 03:56:20,408 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for dc061f98c53d46a6bee0d2161fc0b0e1 2023-02-11 03:56:20,816 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/953c5751d1228a55e688b51c4b8bb6fe/.tmp/family2/81b06e9c3aa54dcca7996da2a54ea990 2023-02-11 03:56:20,819 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 81b06e9c3aa54dcca7996da2a54ea990 2023-02-11 03:56:20,820 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/953c5751d1228a55e688b51c4b8bb6fe/.tmp/family1/77a255e6f2b54178b78a4d3c75f27388 as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/953c5751d1228a55e688b51c4b8bb6fe/family1/77a255e6f2b54178b78a4d3c75f27388 2023-02-11 03:56:20,822 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/30759f6081f5ddc390558cd75a0299b8/.tmp/family2/b57223d55d914ecdb19acd5f1f89ef3d 2023-02-11 03:56:20,825 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for b57223d55d914ecdb19acd5f1f89ef3d 2023-02-11 03:56:20,826 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/30759f6081f5ddc390558cd75a0299b8/.tmp/family1/dc061f98c53d46a6bee0d2161fc0b0e1 as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/30759f6081f5ddc390558cd75a0299b8/family1/dc061f98c53d46a6bee0d2161fc0b0e1 2023-02-11 03:56:20,852 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 77a255e6f2b54178b78a4d3c75f27388 2023-02-11 03:56:20,852 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/953c5751d1228a55e688b51c4b8bb6fe/family1/77a255e6f2b54178b78a4d3c75f27388, entries=3, sequenceid=23, filesize=5.2 K 2023-02-11 03:56:20,853 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/953c5751d1228a55e688b51c4b8bb6fe/.tmp/family2/81b06e9c3aa54dcca7996da2a54ea990 as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/953c5751d1228a55e688b51c4b8bb6fe/family2/81b06e9c3aa54dcca7996da2a54ea990 2023-02-11 03:56:20,855 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for dc061f98c53d46a6bee0d2161fc0b0e1 2023-02-11 03:56:20,855 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/30759f6081f5ddc390558cd75a0299b8/family1/dc061f98c53d46a6bee0d2161fc0b0e1, entries=3, sequenceid=23, filesize=5.2 K 2023-02-11 03:56:20,856 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/30759f6081f5ddc390558cd75a0299b8/.tmp/family2/b57223d55d914ecdb19acd5f1f89ef3d as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/30759f6081f5ddc390558cd75a0299b8/family2/b57223d55d914ecdb19acd5f1f89ef3d 2023-02-11 03:56:20,856 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 81b06e9c3aa54dcca7996da2a54ea990 2023-02-11 03:56:20,856 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/953c5751d1228a55e688b51c4b8bb6fe/family2/81b06e9c3aa54dcca7996da2a54ea990, entries=1, sequenceid=23, filesize=5.0 K 2023-02-11 03:56:20,857 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-1] regionserver.HRegion(2947): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for 953c5751d1228a55e688b51c4b8bb6fe in 1332ms, sequenceid=23, compaction requested=true 2023-02-11 03:56:20,857 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-1] regionserver.HRegion(2445): Flush status journal for 953c5751d1228a55e688b51c4b8bb6fe: 2023-02-11 03:56:20,857 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,,1676087770094.953c5751d1228a55e688b51c4b8bb6fe. 2023-02-11 03:56:20,857 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-02-11 03:56:20,860 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for b57223d55d914ecdb19acd5f1f89ef3d 2023-02-11 03:56:20,860 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/30759f6081f5ddc390558cd75a0299b8/family2/b57223d55d914ecdb19acd5f1f89ef3d, entries=1, sequenceid=23, filesize=5.0 K 2023-02-11 03:56:20,860 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-2] regionserver.HRegion(2947): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for 30759f6081f5ddc390558cd75a0299b8 in 1335ms, sequenceid=23, compaction requested=true 2023-02-11 03:56:20,860 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-2] regionserver.HRegion(2445): Flush status journal for 30759f6081f5ddc390558cd75a0299b8: 2023-02-11 03:56:20,860 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,2,1676087770094.30759f6081f5ddc390558cd75a0299b8. 2023-02-11 03:56:20,860 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-02-11 03:56:21,218 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/fe3a2eacea8dd8daae28302db14142ed/.tmp/family2/dbbdbb80d76e45989290f74f295f066c 2023-02-11 03:56:21,221 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for dbbdbb80d76e45989290f74f295f066c 2023-02-11 03:56:21,222 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/fe3a2eacea8dd8daae28302db14142ed/.tmp/family1/249226ed0d3449ba89686cf1c9713498 as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/fe3a2eacea8dd8daae28302db14142ed/family1/249226ed0d3449ba89686cf1c9713498 2023-02-11 03:56:21,244 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 249226ed0d3449ba89686cf1c9713498 2023-02-11 03:56:21,244 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/fe3a2eacea8dd8daae28302db14142ed/family1/249226ed0d3449ba89686cf1c9713498, entries=3, sequenceid=23, filesize=5.2 K 2023-02-11 03:56:21,245 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/fe3a2eacea8dd8daae28302db14142ed/.tmp/family2/dbbdbb80d76e45989290f74f295f066c as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/fe3a2eacea8dd8daae28302db14142ed/family2/dbbdbb80d76e45989290f74f295f066c 2023-02-11 03:56:21,248 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for dbbdbb80d76e45989290f74f295f066c 2023-02-11 03:56:21,248 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/fe3a2eacea8dd8daae28302db14142ed/family2/dbbdbb80d76e45989290f74f295f066c, entries=1, sequenceid=23, filesize=5.0 K 2023-02-11 03:56:21,249 INFO [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for fe3a2eacea8dd8daae28302db14142ed in 1724ms, sequenceid=23, compaction requested=true 2023-02-11 03:56:21,249 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for fe3a2eacea8dd8daae28302db14142ed: 2023-02-11 03:56:21,249 DEBUG [rs(jenkins-hbase4.apache.org,37011,1676087618428)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,1,1676087770094.fe3a2eacea8dd8daae28302db14142ed. 2023-02-11 03:56:21,249 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-02-11 03:56:21,249 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-02-11 03:56:21,249 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:21,249 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionFromAdmin' locally acquired 2023-02-11 03:56:21,249 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37011,1676087618428' joining acquired barrier for procedure (testMajorCompactionFromAdmin) in zk 2023-02-11 03:56:21,251 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-11 03:56:21,251 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:21,251 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:21,251 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:56:21,251 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:56:21,251 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-11 03:56:21,251 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionFromAdmin' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-11 03:56:21,251 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:56:21,251 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:56:21,252 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-11 03:56:21,252 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:21,252 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:56:21,252 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37011,1676087618428' joining acquired barrier for procedure 'testMajorCompactionFromAdmin' on coordinator 2023-02-11 03:56:21,252 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@60e3bc01[Count = 0] remaining members to acquire global barrier 2023-02-11 03:56:21,252 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionFromAdmin' starting 'in-barrier' execution. 2023-02-11 03:56:21,252 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-11 03:56:21,253 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-11 03:56:21,253 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-11 03:56:21,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-11 03:56:21,254 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionFromAdmin' received 'reached' from coordinator. 2023-02-11 03:56:21,254 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:21,254 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-11 03:56:21,254 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionFromAdmin' locally completed 2023-02-11 03:56:21,254 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionFromAdmin' completed for member 'jenkins-hbase4.apache.org,37011,1676087618428' in zk 2023-02-11 03:56:21,255 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:21,255 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionFromAdmin' has notified controller of completion 2023-02-11 03:56:21,255 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:21,255 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:56:21,255 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:56:21,255 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:56:21,255 DEBUG [member: 'jenkins-hbase4.apache.org,37011,1676087618428' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionFromAdmin' completed. 2023-02-11 03:56:21,255 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:56:21,255 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:56:21,256 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-11 03:56:21,256 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:21,256 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:56:21,256 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-11 03:56:21,257 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:21,257 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionFromAdmin' member 'jenkins-hbase4.apache.org,37011,1676087618428': 2023-02-11 03:56:21,257 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37011,1676087618428' released barrier for procedure'testMajorCompactionFromAdmin', counting down latch. Waiting for 0 more 2023-02-11 03:56:21,257 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionFromAdmin' execution completed 2023-02-11 03:56:21,257 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-11 03:56:21,257 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-11 03:56:21,257 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionFromAdmin 2023-02-11 03:56:21,257 INFO [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionFromAdminincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-11 03:56:21,259 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-11 03:56:21,259 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:56:21,259 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-11 03:56:21,259 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-11 03:56:21,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-11 03:56:21,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-11 03:56:21,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-11 03:56:21,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-11 03:56:21,260 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:56:21,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:56:21,260 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:21,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-11 03:56:21,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-11 03:56:21,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-11 03:56:21,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-11 03:56:21,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-11 03:56:21,261 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:21,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:21,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-11 03:56:21,262 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-11 03:56:21,262 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:21,263 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:21,263 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-11 03:56:21,263 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-02-11 03:56:21,263 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-11 03:56:21,263 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39921] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionFromAdmin' 2023-02-11 03:56:21,263 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39921] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-11 03:56:21,263 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-11 03:56:21,263 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-02-11 03:56:21,263 DEBUG [(jenkins-hbase4.apache.org,39921,1676087617326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-11 03:56:21,263 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:21,263 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-11 03:56:21,264 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-11 03:56:21,264 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-11 03:56:21,263 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin because node does not exist (not an error) 2023-02-11 03:56:21,264 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-11 03:56:21,264 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionFromAdmin'' to complete. (max 300000 ms per retry) 2023-02-11 03:56:21,264 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-11 03:56:21,264 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-11 03:56:21,264 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-11 03:56:21,364 DEBUG [Listener at localhost/46205] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-11 03:56:21,365 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39921] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-11 03:56:21,365 DEBUG [Listener at localhost/46205] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-02-11 03:56:21,366 DEBUG [Listener at localhost/46205] regionserver.HStore(1912): fe3a2eacea8dd8daae28302db14142ed/family1 is initiating major compaction (all files) 2023-02-11 03:56:21,366 INFO [Listener at localhost/46205] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-11 03:56:21,366 INFO [Listener at localhost/46205] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-11 03:56:21,368 INFO [Listener at localhost/46205] regionserver.HRegion(2258): Starting compaction of fe3a2eacea8dd8daae28302db14142ed/family1 in testMajorCompactionFromAdmin,1,1676087770094.fe3a2eacea8dd8daae28302db14142ed. 2023-02-11 03:56:21,368 INFO [Listener at localhost/46205] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/fe3a2eacea8dd8daae28302db14142ed/family1/e7cf10565b304f34b02db69e5b8863cc, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/fe3a2eacea8dd8daae28302db14142ed/family1/98f3723d9f6a461b8d01e0758908e398, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/fe3a2eacea8dd8daae28302db14142ed/family1/a70037e87d3345589853ba1104a091c7, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/fe3a2eacea8dd8daae28302db14142ed/family1/90203d6e75a841a6a5d172b0b422cd9e, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/fe3a2eacea8dd8daae28302db14142ed/family1/249226ed0d3449ba89686cf1c9713498] into tmpdir=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/fe3a2eacea8dd8daae28302db14142ed/.tmp, totalSize=27.1 K 2023-02-11 03:56:21,368 DEBUG [Listener at localhost/46205] compactions.Compactor(207): Compacting e7cf10565b304f34b02db69e5b8863cc, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1676087772238 2023-02-11 03:56:21,369 DEBUG [Listener at localhost/46205] compactions.Compactor(207): Compacting 98f3723d9f6a461b8d01e0758908e398, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1676087774119 2023-02-11 03:56:21,369 DEBUG [Listener at localhost/46205] compactions.Compactor(207): Compacting a70037e87d3345589853ba1104a091c7, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1676087776008 2023-02-11 03:56:21,369 DEBUG [Listener at localhost/46205] compactions.Compactor(207): Compacting 90203d6e75a841a6a5d172b0b422cd9e, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1676087777514 2023-02-11 03:56:21,370 DEBUG [Listener at localhost/46205] compactions.Compactor(207): Compacting 249226ed0d3449ba89686cf1c9713498, keycount=3, bloomtype=ROW, size=5.2 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-02-11 03:56:21,381 INFO [Listener at localhost/46205] throttle.PressureAwareThroughputController(145): fe3a2eacea8dd8daae28302db14142ed#family1#compaction#197 average throughput is 0.86 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-11 03:56:21,819 INFO [Listener at localhost/46205] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b202302115a6a6fe96a334d89b2b2c7e28a56370f_del 2023-02-11 03:56:21,819 INFO [Listener at localhost/46205] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202302115a6a6fe96a334d89b2b2c7e28a56370f_del to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b202302115a6a6fe96a334d89b2b2c7e28a56370f_del 2023-02-11 03:56:22,671 INFO [Listener at localhost/46205] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d0fc09a041534eb2b63636b8a4a29bc8 2023-02-11 03:56:22,671 DEBUG [Listener at localhost/46205] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/fe3a2eacea8dd8daae28302db14142ed/.tmp/family1/d0fc09a041534eb2b63636b8a4a29bc8 as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/fe3a2eacea8dd8daae28302db14142ed/family1/d0fc09a041534eb2b63636b8a4a29bc8 2023-02-11 03:56:22,693 INFO [Listener at localhost/46205] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d0fc09a041534eb2b63636b8a4a29bc8 2023-02-11 03:56:22,694 INFO [Listener at localhost/46205] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in fe3a2eacea8dd8daae28302db14142ed/family1 of fe3a2eacea8dd8daae28302db14142ed into d0fc09a041534eb2b63636b8a4a29bc8(size=7.0 K), total size for store is 7.0 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-02-11 03:56:22,694 DEBUG [Listener at localhost/46205] regionserver.HRegion(2288): Compaction status journal for fe3a2eacea8dd8daae28302db14142ed: 2023-02-11 03:56:22,694 DEBUG [Listener at localhost/46205] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-02-11 03:56:22,695 DEBUG [Listener at localhost/46205] regionserver.HStore(1912): fe3a2eacea8dd8daae28302db14142ed/family2 is initiating major compaction (all files) 2023-02-11 03:56:22,695 INFO [Listener at localhost/46205] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-11 03:56:22,695 INFO [Listener at localhost/46205] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-11 03:56:22,695 INFO [Listener at localhost/46205] regionserver.HRegion(2258): Starting compaction of fe3a2eacea8dd8daae28302db14142ed/family2 in testMajorCompactionFromAdmin,1,1676087770094.fe3a2eacea8dd8daae28302db14142ed. 2023-02-11 03:56:22,695 INFO [Listener at localhost/46205] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/fe3a2eacea8dd8daae28302db14142ed/family2/0362b086d5c94dec90e6cd8d7edeedcf, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/fe3a2eacea8dd8daae28302db14142ed/family2/9ac549495c004b308436f6f6d1e539d7, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/fe3a2eacea8dd8daae28302db14142ed/family2/4341eff9ec7443839ebc21d9e8f0d8a4, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/fe3a2eacea8dd8daae28302db14142ed/family2/a1de535155ef4b41b92da28d66e43a64, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/fe3a2eacea8dd8daae28302db14142ed/family2/dbbdbb80d76e45989290f74f295f066c] into tmpdir=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/fe3a2eacea8dd8daae28302db14142ed/.tmp, totalSize=25.4 K 2023-02-11 03:56:22,695 DEBUG [Listener at localhost/46205] compactions.Compactor(207): Compacting 0362b086d5c94dec90e6cd8d7edeedcf, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1676087772238 2023-02-11 03:56:22,696 DEBUG [Listener at localhost/46205] compactions.Compactor(207): Compacting 9ac549495c004b308436f6f6d1e539d7, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1676087774119 2023-02-11 03:56:22,696 DEBUG [Listener at localhost/46205] compactions.Compactor(207): Compacting 4341eff9ec7443839ebc21d9e8f0d8a4, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1676087776008 2023-02-11 03:56:22,696 DEBUG [Listener at localhost/46205] compactions.Compactor(207): Compacting a1de535155ef4b41b92da28d66e43a64, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1676087777514 2023-02-11 03:56:22,697 DEBUG [Listener at localhost/46205] compactions.Compactor(207): Compacting dbbdbb80d76e45989290f74f295f066c, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-02-11 03:56:22,703 INFO [Listener at localhost/46205] throttle.PressureAwareThroughputController(145): fe3a2eacea8dd8daae28302db14142ed#family2#compaction#198 average throughput is 1.06 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-11 03:56:22,753 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/fe3a2eacea8dd8daae28302db14142ed/family1/e7cf10565b304f34b02db69e5b8863cc, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/fe3a2eacea8dd8daae28302db14142ed/family1/98f3723d9f6a461b8d01e0758908e398, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/fe3a2eacea8dd8daae28302db14142ed/family1/a70037e87d3345589853ba1104a091c7, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/fe3a2eacea8dd8daae28302db14142ed/family1/90203d6e75a841a6a5d172b0b422cd9e, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/fe3a2eacea8dd8daae28302db14142ed/family1/249226ed0d3449ba89686cf1c9713498] to archive 2023-02-11 03:56:22,754 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-11 03:56:22,756 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/fe3a2eacea8dd8daae28302db14142ed/family1/e7cf10565b304f34b02db69e5b8863cc to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testMajorCompactionFromAdmin/fe3a2eacea8dd8daae28302db14142ed/family1/e7cf10565b304f34b02db69e5b8863cc 2023-02-11 03:56:22,757 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/fe3a2eacea8dd8daae28302db14142ed/family1/98f3723d9f6a461b8d01e0758908e398 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testMajorCompactionFromAdmin/fe3a2eacea8dd8daae28302db14142ed/family1/98f3723d9f6a461b8d01e0758908e398 2023-02-11 03:56:22,758 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/fe3a2eacea8dd8daae28302db14142ed/family1/a70037e87d3345589853ba1104a091c7 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testMajorCompactionFromAdmin/fe3a2eacea8dd8daae28302db14142ed/family1/a70037e87d3345589853ba1104a091c7 2023-02-11 03:56:22,759 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/fe3a2eacea8dd8daae28302db14142ed/family1/90203d6e75a841a6a5d172b0b422cd9e to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testMajorCompactionFromAdmin/fe3a2eacea8dd8daae28302db14142ed/family1/90203d6e75a841a6a5d172b0b422cd9e 2023-02-11 03:56:22,760 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/fe3a2eacea8dd8daae28302db14142ed/family1/249226ed0d3449ba89686cf1c9713498 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testMajorCompactionFromAdmin/fe3a2eacea8dd8daae28302db14142ed/family1/249226ed0d3449ba89686cf1c9713498 2023-02-11 03:56:23,111 INFO [Listener at localhost/46205] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b20230211a2177eaf5f784b2c9aadafc5f8fffa20_del 2023-02-11 03:56:23,111 INFO [Listener at localhost/46205] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230211a2177eaf5f784b2c9aadafc5f8fffa20_del to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c4ca4238a0b923820dcc509a6f75849b20230211a2177eaf5f784b2c9aadafc5f8fffa20_del 2023-02-11 03:56:23,924 INFO [Listener at localhost/46205] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 8c4de9ad38c0482f9100fc920f5a54f7 2023-02-11 03:56:23,924 DEBUG [Listener at localhost/46205] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/fe3a2eacea8dd8daae28302db14142ed/.tmp/family2/8c4de9ad38c0482f9100fc920f5a54f7 as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/fe3a2eacea8dd8daae28302db14142ed/family2/8c4de9ad38c0482f9100fc920f5a54f7 2023-02-11 03:56:23,927 INFO [Listener at localhost/46205] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 8c4de9ad38c0482f9100fc920f5a54f7 2023-02-11 03:56:23,928 INFO [Listener at localhost/46205] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in fe3a2eacea8dd8daae28302db14142ed/family2 of fe3a2eacea8dd8daae28302db14142ed into 8c4de9ad38c0482f9100fc920f5a54f7(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-02-11 03:56:23,928 DEBUG [Listener at localhost/46205] regionserver.HRegion(2288): Compaction status journal for fe3a2eacea8dd8daae28302db14142ed: 2023-02-11 03:56:23,928 DEBUG [Listener at localhost/46205] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-02-11 03:56:23,929 DEBUG [Listener at localhost/46205] regionserver.HStore(1912): 953c5751d1228a55e688b51c4b8bb6fe/family1 is initiating major compaction (all files) 2023-02-11 03:56:23,929 INFO [Listener at localhost/46205] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-11 03:56:23,929 INFO [Listener at localhost/46205] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-11 03:56:23,929 INFO [Listener at localhost/46205] regionserver.HRegion(2258): Starting compaction of 953c5751d1228a55e688b51c4b8bb6fe/family1 in testMajorCompactionFromAdmin,,1676087770094.953c5751d1228a55e688b51c4b8bb6fe. 2023-02-11 03:56:23,929 INFO [Listener at localhost/46205] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/953c5751d1228a55e688b51c4b8bb6fe/family1/9bcd485d965341ed8dab23d2ff57c13d, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/953c5751d1228a55e688b51c4b8bb6fe/family1/fd99ec7b081c47b0b139a5010d997ac4, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/953c5751d1228a55e688b51c4b8bb6fe/family1/f763e1f9e5484f0483d251fc2b86b0e7, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/953c5751d1228a55e688b51c4b8bb6fe/family1/09fb95e4f1e74b45b68b3aacca080f46, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/953c5751d1228a55e688b51c4b8bb6fe/family1/77a255e6f2b54178b78a4d3c75f27388] into tmpdir=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/953c5751d1228a55e688b51c4b8bb6fe/.tmp, totalSize=27.1 K 2023-02-11 03:56:23,930 DEBUG [Listener at localhost/46205] compactions.Compactor(207): Compacting 9bcd485d965341ed8dab23d2ff57c13d, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1676087772237 2023-02-11 03:56:23,930 DEBUG [Listener at localhost/46205] compactions.Compactor(207): Compacting fd99ec7b081c47b0b139a5010d997ac4, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1676087774118 2023-02-11 03:56:23,930 DEBUG [Listener at localhost/46205] compactions.Compactor(207): Compacting f763e1f9e5484f0483d251fc2b86b0e7, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1676087776008 2023-02-11 03:56:23,931 DEBUG [Listener at localhost/46205] compactions.Compactor(207): Compacting 09fb95e4f1e74b45b68b3aacca080f46, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1676087777514 2023-02-11 03:56:23,931 DEBUG [Listener at localhost/46205] compactions.Compactor(207): Compacting 77a255e6f2b54178b78a4d3c75f27388, keycount=3, bloomtype=ROW, size=5.2 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-02-11 03:56:23,938 INFO [Listener at localhost/46205] throttle.PressureAwareThroughputController(145): 953c5751d1228a55e688b51c4b8bb6fe#family1#compaction#199 average throughput is 0.86 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-11 03:56:23,950 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/fe3a2eacea8dd8daae28302db14142ed/family2/0362b086d5c94dec90e6cd8d7edeedcf, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/fe3a2eacea8dd8daae28302db14142ed/family2/9ac549495c004b308436f6f6d1e539d7, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/fe3a2eacea8dd8daae28302db14142ed/family2/4341eff9ec7443839ebc21d9e8f0d8a4, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/fe3a2eacea8dd8daae28302db14142ed/family2/a1de535155ef4b41b92da28d66e43a64, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/fe3a2eacea8dd8daae28302db14142ed/family2/dbbdbb80d76e45989290f74f295f066c] to archive 2023-02-11 03:56:23,950 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-11 03:56:23,952 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/fe3a2eacea8dd8daae28302db14142ed/family2/0362b086d5c94dec90e6cd8d7edeedcf to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testMajorCompactionFromAdmin/fe3a2eacea8dd8daae28302db14142ed/family2/0362b086d5c94dec90e6cd8d7edeedcf 2023-02-11 03:56:23,953 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/fe3a2eacea8dd8daae28302db14142ed/family2/9ac549495c004b308436f6f6d1e539d7 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testMajorCompactionFromAdmin/fe3a2eacea8dd8daae28302db14142ed/family2/9ac549495c004b308436f6f6d1e539d7 2023-02-11 03:56:23,954 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/fe3a2eacea8dd8daae28302db14142ed/family2/4341eff9ec7443839ebc21d9e8f0d8a4 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testMajorCompactionFromAdmin/fe3a2eacea8dd8daae28302db14142ed/family2/4341eff9ec7443839ebc21d9e8f0d8a4 2023-02-11 03:56:23,955 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/fe3a2eacea8dd8daae28302db14142ed/family2/a1de535155ef4b41b92da28d66e43a64 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testMajorCompactionFromAdmin/fe3a2eacea8dd8daae28302db14142ed/family2/a1de535155ef4b41b92da28d66e43a64 2023-02-11 03:56:23,956 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/fe3a2eacea8dd8daae28302db14142ed/family2/dbbdbb80d76e45989290f74f295f066c to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testMajorCompactionFromAdmin/fe3a2eacea8dd8daae28302db14142ed/family2/dbbdbb80d76e45989290f74f295f066c 2023-02-11 03:56:23,974 INFO [Listener at localhost/46205] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e202302112ba5c8b097424919ae112aa728d151c9_del 2023-02-11 03:56:23,974 INFO [Listener at localhost/46205] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202302112ba5c8b097424919ae112aa728d151c9_del to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e202302112ba5c8b097424919ae112aa728d151c9_del 2023-02-11 03:56:24,826 INFO [Listener at localhost/46205] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 3bb5b02500294742987d2551849a5a0d 2023-02-11 03:56:24,827 DEBUG [Listener at localhost/46205] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/953c5751d1228a55e688b51c4b8bb6fe/.tmp/family1/3bb5b02500294742987d2551849a5a0d as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/953c5751d1228a55e688b51c4b8bb6fe/family1/3bb5b02500294742987d2551849a5a0d 2023-02-11 03:56:24,848 INFO [Listener at localhost/46205] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 3bb5b02500294742987d2551849a5a0d 2023-02-11 03:56:24,849 INFO [Listener at localhost/46205] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 953c5751d1228a55e688b51c4b8bb6fe/family1 of 953c5751d1228a55e688b51c4b8bb6fe into 3bb5b02500294742987d2551849a5a0d(size=7.0 K), total size for store is 7.0 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-11 03:56:24,849 DEBUG [Listener at localhost/46205] regionserver.HRegion(2288): Compaction status journal for 953c5751d1228a55e688b51c4b8bb6fe: 2023-02-11 03:56:24,849 DEBUG [Listener at localhost/46205] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-02-11 03:56:24,849 DEBUG [Listener at localhost/46205] regionserver.HStore(1912): 953c5751d1228a55e688b51c4b8bb6fe/family2 is initiating major compaction (all files) 2023-02-11 03:56:24,849 INFO [Listener at localhost/46205] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-11 03:56:24,850 INFO [Listener at localhost/46205] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-11 03:56:24,850 INFO [Listener at localhost/46205] regionserver.HRegion(2258): Starting compaction of 953c5751d1228a55e688b51c4b8bb6fe/family2 in testMajorCompactionFromAdmin,,1676087770094.953c5751d1228a55e688b51c4b8bb6fe. 2023-02-11 03:56:24,850 INFO [Listener at localhost/46205] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/953c5751d1228a55e688b51c4b8bb6fe/family2/f7e13c216d95421c832d580d536a91b4, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/953c5751d1228a55e688b51c4b8bb6fe/family2/688326961d3a40008a354222d19ec75c, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/953c5751d1228a55e688b51c4b8bb6fe/family2/548fe02449c644c49b4e45e3c789ec3e, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/953c5751d1228a55e688b51c4b8bb6fe/family2/df4b77f4363d4fe3b8d60b17ef71d390, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/953c5751d1228a55e688b51c4b8bb6fe/family2/81b06e9c3aa54dcca7996da2a54ea990] into tmpdir=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/953c5751d1228a55e688b51c4b8bb6fe/.tmp, totalSize=25.4 K 2023-02-11 03:56:24,850 DEBUG [Listener at localhost/46205] compactions.Compactor(207): Compacting f7e13c216d95421c832d580d536a91b4, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1676087772237 2023-02-11 03:56:24,850 DEBUG [Listener at localhost/46205] compactions.Compactor(207): Compacting 688326961d3a40008a354222d19ec75c, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1676087774118 2023-02-11 03:56:24,851 DEBUG [Listener at localhost/46205] compactions.Compactor(207): Compacting 548fe02449c644c49b4e45e3c789ec3e, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1676087776008 2023-02-11 03:56:24,851 DEBUG [Listener at localhost/46205] compactions.Compactor(207): Compacting df4b77f4363d4fe3b8d60b17ef71d390, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1676087777514 2023-02-11 03:56:24,851 DEBUG [Listener at localhost/46205] compactions.Compactor(207): Compacting 81b06e9c3aa54dcca7996da2a54ea990, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-02-11 03:56:24,854 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/953c5751d1228a55e688b51c4b8bb6fe/family1/9bcd485d965341ed8dab23d2ff57c13d, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/953c5751d1228a55e688b51c4b8bb6fe/family1/fd99ec7b081c47b0b139a5010d997ac4, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/953c5751d1228a55e688b51c4b8bb6fe/family1/f763e1f9e5484f0483d251fc2b86b0e7, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/953c5751d1228a55e688b51c4b8bb6fe/family1/09fb95e4f1e74b45b68b3aacca080f46, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/953c5751d1228a55e688b51c4b8bb6fe/family1/77a255e6f2b54178b78a4d3c75f27388] to archive 2023-02-11 03:56:24,854 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-11 03:56:24,856 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/953c5751d1228a55e688b51c4b8bb6fe/family1/9bcd485d965341ed8dab23d2ff57c13d to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testMajorCompactionFromAdmin/953c5751d1228a55e688b51c4b8bb6fe/family1/9bcd485d965341ed8dab23d2ff57c13d 2023-02-11 03:56:24,857 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/953c5751d1228a55e688b51c4b8bb6fe/family1/fd99ec7b081c47b0b139a5010d997ac4 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testMajorCompactionFromAdmin/953c5751d1228a55e688b51c4b8bb6fe/family1/fd99ec7b081c47b0b139a5010d997ac4 2023-02-11 03:56:24,858 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/953c5751d1228a55e688b51c4b8bb6fe/family1/f763e1f9e5484f0483d251fc2b86b0e7 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testMajorCompactionFromAdmin/953c5751d1228a55e688b51c4b8bb6fe/family1/f763e1f9e5484f0483d251fc2b86b0e7 2023-02-11 03:56:24,859 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/953c5751d1228a55e688b51c4b8bb6fe/family1/09fb95e4f1e74b45b68b3aacca080f46 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testMajorCompactionFromAdmin/953c5751d1228a55e688b51c4b8bb6fe/family1/09fb95e4f1e74b45b68b3aacca080f46 2023-02-11 03:56:24,859 INFO [Listener at localhost/46205] throttle.PressureAwareThroughputController(145): 953c5751d1228a55e688b51c4b8bb6fe#family2#compaction#200 average throughput is 1.06 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-11 03:56:24,860 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/953c5751d1228a55e688b51c4b8bb6fe/family1/77a255e6f2b54178b78a4d3c75f27388 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testMajorCompactionFromAdmin/953c5751d1228a55e688b51c4b8bb6fe/family1/77a255e6f2b54178b78a4d3c75f27388 2023-02-11 03:56:25,273 INFO [Listener at localhost/46205] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e20230211630c6532f1e54ce7a3a2f246c01e9c5e_del 2023-02-11 03:56:25,273 INFO [Listener at localhost/46205] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230211630c6532f1e54ce7a3a2f246c01e9c5e_del to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/d41d8cd98f00b204e9800998ecf8427e20230211630c6532f1e54ce7a3a2f246c01e9c5e_del 2023-02-11 03:56:26,085 INFO [Listener at localhost/46205] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 6ce31a53f267432eb356f0756fa7ad22 2023-02-11 03:56:26,086 DEBUG [Listener at localhost/46205] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/953c5751d1228a55e688b51c4b8bb6fe/.tmp/family2/6ce31a53f267432eb356f0756fa7ad22 as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/953c5751d1228a55e688b51c4b8bb6fe/family2/6ce31a53f267432eb356f0756fa7ad22 2023-02-11 03:56:26,089 INFO [Listener at localhost/46205] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 6ce31a53f267432eb356f0756fa7ad22 2023-02-11 03:56:26,090 INFO [Listener at localhost/46205] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 953c5751d1228a55e688b51c4b8bb6fe/family2 of 953c5751d1228a55e688b51c4b8bb6fe into 6ce31a53f267432eb356f0756fa7ad22(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-02-11 03:56:26,091 DEBUG [Listener at localhost/46205] regionserver.HRegion(2288): Compaction status journal for 953c5751d1228a55e688b51c4b8bb6fe: 2023-02-11 03:56:26,091 DEBUG [Listener at localhost/46205] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-02-11 03:56:26,091 DEBUG [Listener at localhost/46205] regionserver.HStore(1912): 30759f6081f5ddc390558cd75a0299b8/family1 is initiating major compaction (all files) 2023-02-11 03:56:26,091 INFO [Listener at localhost/46205] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-11 03:56:26,091 INFO [Listener at localhost/46205] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-11 03:56:26,091 INFO [Listener at localhost/46205] regionserver.HRegion(2258): Starting compaction of 30759f6081f5ddc390558cd75a0299b8/family1 in testMajorCompactionFromAdmin,2,1676087770094.30759f6081f5ddc390558cd75a0299b8. 2023-02-11 03:56:26,091 INFO [Listener at localhost/46205] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/30759f6081f5ddc390558cd75a0299b8/family1/394aad0fbd074d8093fb08363fb9a75f, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/30759f6081f5ddc390558cd75a0299b8/family1/0c688f0c757149c789c43cea2c343feb, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/30759f6081f5ddc390558cd75a0299b8/family1/bb80ff603e2e49ddaba5457a7cd04a14, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/30759f6081f5ddc390558cd75a0299b8/family1/a27b5488126c4e97847bea80eeb14cea, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/30759f6081f5ddc390558cd75a0299b8/family1/dc061f98c53d46a6bee0d2161fc0b0e1] into tmpdir=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/30759f6081f5ddc390558cd75a0299b8/.tmp, totalSize=27.1 K 2023-02-11 03:56:26,091 DEBUG [Listener at localhost/46205] compactions.Compactor(207): Compacting 394aad0fbd074d8093fb08363fb9a75f, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1676087772238 2023-02-11 03:56:26,092 DEBUG [Listener at localhost/46205] compactions.Compactor(207): Compacting 0c688f0c757149c789c43cea2c343feb, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1676087774119 2023-02-11 03:56:26,092 DEBUG [Listener at localhost/46205] compactions.Compactor(207): Compacting bb80ff603e2e49ddaba5457a7cd04a14, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1676087776009 2023-02-11 03:56:26,092 DEBUG [Listener at localhost/46205] compactions.Compactor(207): Compacting a27b5488126c4e97847bea80eeb14cea, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1676087777514 2023-02-11 03:56:26,093 DEBUG [Listener at localhost/46205] compactions.Compactor(207): Compacting dc061f98c53d46a6bee0d2161fc0b0e1, keycount=3, bloomtype=ROW, size=5.2 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-02-11 03:56:26,100 INFO [Listener at localhost/46205] throttle.PressureAwareThroughputController(145): 30759f6081f5ddc390558cd75a0299b8#family1#compaction#201 average throughput is 0.86 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-11 03:56:26,149 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/953c5751d1228a55e688b51c4b8bb6fe/family2/f7e13c216d95421c832d580d536a91b4, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/953c5751d1228a55e688b51c4b8bb6fe/family2/688326961d3a40008a354222d19ec75c, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/953c5751d1228a55e688b51c4b8bb6fe/family2/548fe02449c644c49b4e45e3c789ec3e, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/953c5751d1228a55e688b51c4b8bb6fe/family2/df4b77f4363d4fe3b8d60b17ef71d390, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/953c5751d1228a55e688b51c4b8bb6fe/family2/81b06e9c3aa54dcca7996da2a54ea990] to archive 2023-02-11 03:56:26,150 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-11 03:56:26,152 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/953c5751d1228a55e688b51c4b8bb6fe/family2/f7e13c216d95421c832d580d536a91b4 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testMajorCompactionFromAdmin/953c5751d1228a55e688b51c4b8bb6fe/family2/f7e13c216d95421c832d580d536a91b4 2023-02-11 03:56:26,153 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/953c5751d1228a55e688b51c4b8bb6fe/family2/688326961d3a40008a354222d19ec75c to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testMajorCompactionFromAdmin/953c5751d1228a55e688b51c4b8bb6fe/family2/688326961d3a40008a354222d19ec75c 2023-02-11 03:56:26,154 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/953c5751d1228a55e688b51c4b8bb6fe/family2/548fe02449c644c49b4e45e3c789ec3e to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testMajorCompactionFromAdmin/953c5751d1228a55e688b51c4b8bb6fe/family2/548fe02449c644c49b4e45e3c789ec3e 2023-02-11 03:56:26,154 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/953c5751d1228a55e688b51c4b8bb6fe/family2/df4b77f4363d4fe3b8d60b17ef71d390 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testMajorCompactionFromAdmin/953c5751d1228a55e688b51c4b8bb6fe/family2/df4b77f4363d4fe3b8d60b17ef71d390 2023-02-11 03:56:26,155 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/953c5751d1228a55e688b51c4b8bb6fe/family2/81b06e9c3aa54dcca7996da2a54ea990 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testMajorCompactionFromAdmin/953c5751d1228a55e688b51c4b8bb6fe/family2/81b06e9c3aa54dcca7996da2a54ea990 2023-02-11 03:56:26,535 INFO [Listener at localhost/46205] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c202302115664a3bfb3b14bfbbd8fe7c6c8ff7802_del 2023-02-11 03:56:26,535 INFO [Listener at localhost/46205] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202302115664a3bfb3b14bfbbd8fe7c6c8ff7802_del to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c202302115664a3bfb3b14bfbbd8fe7c6c8ff7802_del 2023-02-11 03:56:27,389 INFO [Listener at localhost/46205] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 03ca7d1c6ccc48eb8876e10a5d04b8c1 2023-02-11 03:56:27,390 DEBUG [Listener at localhost/46205] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/30759f6081f5ddc390558cd75a0299b8/.tmp/family1/03ca7d1c6ccc48eb8876e10a5d04b8c1 as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/30759f6081f5ddc390558cd75a0299b8/family1/03ca7d1c6ccc48eb8876e10a5d04b8c1 2023-02-11 03:56:27,412 INFO [Listener at localhost/46205] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 03ca7d1c6ccc48eb8876e10a5d04b8c1 2023-02-11 03:56:27,413 INFO [Listener at localhost/46205] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 30759f6081f5ddc390558cd75a0299b8/family1 of 30759f6081f5ddc390558cd75a0299b8 into 03ca7d1c6ccc48eb8876e10a5d04b8c1(size=7.0 K), total size for store is 7.0 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-02-11 03:56:27,413 DEBUG [Listener at localhost/46205] regionserver.HRegion(2288): Compaction status journal for 30759f6081f5ddc390558cd75a0299b8: 2023-02-11 03:56:27,413 DEBUG [Listener at localhost/46205] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-02-11 03:56:27,413 DEBUG [Listener at localhost/46205] regionserver.HStore(1912): 30759f6081f5ddc390558cd75a0299b8/family2 is initiating major compaction (all files) 2023-02-11 03:56:27,413 INFO [Listener at localhost/46205] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-11 03:56:27,413 INFO [Listener at localhost/46205] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-11 03:56:27,413 INFO [Listener at localhost/46205] regionserver.HRegion(2258): Starting compaction of 30759f6081f5ddc390558cd75a0299b8/family2 in testMajorCompactionFromAdmin,2,1676087770094.30759f6081f5ddc390558cd75a0299b8. 2023-02-11 03:56:27,414 INFO [Listener at localhost/46205] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/30759f6081f5ddc390558cd75a0299b8/family2/9d98065c542c4813a153663332fc15e4, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/30759f6081f5ddc390558cd75a0299b8/family2/72fa679c42fa42ed913c4140d456aeed, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/30759f6081f5ddc390558cd75a0299b8/family2/0b1c2c1de15a415488c64db1e9c2a924, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/30759f6081f5ddc390558cd75a0299b8/family2/714b21396662486ea76d3db63c3145ea, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/30759f6081f5ddc390558cd75a0299b8/family2/b57223d55d914ecdb19acd5f1f89ef3d] into tmpdir=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/30759f6081f5ddc390558cd75a0299b8/.tmp, totalSize=25.4 K 2023-02-11 03:56:27,414 DEBUG [Listener at localhost/46205] compactions.Compactor(207): Compacting 9d98065c542c4813a153663332fc15e4, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1676087772238 2023-02-11 03:56:27,414 DEBUG [Listener at localhost/46205] compactions.Compactor(207): Compacting 72fa679c42fa42ed913c4140d456aeed, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1676087774119 2023-02-11 03:56:27,415 DEBUG [Listener at localhost/46205] compactions.Compactor(207): Compacting 0b1c2c1de15a415488c64db1e9c2a924, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1676087776009 2023-02-11 03:56:27,415 DEBUG [Listener at localhost/46205] compactions.Compactor(207): Compacting 714b21396662486ea76d3db63c3145ea, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1676087777514 2023-02-11 03:56:27,415 DEBUG [Listener at localhost/46205] compactions.Compactor(207): Compacting b57223d55d914ecdb19acd5f1f89ef3d, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-02-11 03:56:27,422 INFO [Listener at localhost/46205] throttle.PressureAwareThroughputController(145): 30759f6081f5ddc390558cd75a0299b8#family2#compaction#202 average throughput is 0.53 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-11 03:56:27,453 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/30759f6081f5ddc390558cd75a0299b8/family1/394aad0fbd074d8093fb08363fb9a75f, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/30759f6081f5ddc390558cd75a0299b8/family1/0c688f0c757149c789c43cea2c343feb, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/30759f6081f5ddc390558cd75a0299b8/family1/bb80ff603e2e49ddaba5457a7cd04a14, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/30759f6081f5ddc390558cd75a0299b8/family1/a27b5488126c4e97847bea80eeb14cea, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/30759f6081f5ddc390558cd75a0299b8/family1/dc061f98c53d46a6bee0d2161fc0b0e1] to archive 2023-02-11 03:56:27,453 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-11 03:56:27,454 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/30759f6081f5ddc390558cd75a0299b8/family1/394aad0fbd074d8093fb08363fb9a75f to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testMajorCompactionFromAdmin/30759f6081f5ddc390558cd75a0299b8/family1/394aad0fbd074d8093fb08363fb9a75f 2023-02-11 03:56:27,455 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/30759f6081f5ddc390558cd75a0299b8/family1/0c688f0c757149c789c43cea2c343feb to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testMajorCompactionFromAdmin/30759f6081f5ddc390558cd75a0299b8/family1/0c688f0c757149c789c43cea2c343feb 2023-02-11 03:56:27,456 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/30759f6081f5ddc390558cd75a0299b8/family1/bb80ff603e2e49ddaba5457a7cd04a14 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testMajorCompactionFromAdmin/30759f6081f5ddc390558cd75a0299b8/family1/bb80ff603e2e49ddaba5457a7cd04a14 2023-02-11 03:56:27,457 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/30759f6081f5ddc390558cd75a0299b8/family1/a27b5488126c4e97847bea80eeb14cea to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testMajorCompactionFromAdmin/30759f6081f5ddc390558cd75a0299b8/family1/a27b5488126c4e97847bea80eeb14cea 2023-02-11 03:56:27,458 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/30759f6081f5ddc390558cd75a0299b8/family1/dc061f98c53d46a6bee0d2161fc0b0e1 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testMajorCompactionFromAdmin/30759f6081f5ddc390558cd75a0299b8/family1/dc061f98c53d46a6bee0d2161fc0b0e1 2023-02-11 03:56:27,829 INFO [Listener at localhost/46205] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c20230211978539d1b7de44028e45ed461b39a475_del 2023-02-11 03:56:27,829 INFO [Listener at localhost/46205] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230211978539d1b7de44028e45ed461b39a475_del to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c81e728d9d4c2f636f067f89cc14862c20230211978539d1b7de44028e45ed461b39a475_del 2023-02-11 03:56:28,642 INFO [Listener at localhost/46205] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 08661af8ce6444d79f8c2c4a48d37481 2023-02-11 03:56:28,642 DEBUG [Listener at localhost/46205] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/30759f6081f5ddc390558cd75a0299b8/.tmp/family2/08661af8ce6444d79f8c2c4a48d37481 as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/30759f6081f5ddc390558cd75a0299b8/family2/08661af8ce6444d79f8c2c4a48d37481 2023-02-11 03:56:28,645 INFO [Listener at localhost/46205] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 08661af8ce6444d79f8c2c4a48d37481 2023-02-11 03:56:28,646 INFO [Listener at localhost/46205] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 30759f6081f5ddc390558cd75a0299b8/family2 of 30759f6081f5ddc390558cd75a0299b8 into 08661af8ce6444d79f8c2c4a48d37481(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-02-11 03:56:28,646 DEBUG [Listener at localhost/46205] regionserver.HRegion(2288): Compaction status journal for 30759f6081f5ddc390558cd75a0299b8: 2023-02-11 03:56:28,649 DEBUG [Listener at localhost/46205] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-02-11 03:56:28,651 DEBUG [Listener at localhost/46205] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-02-11 03:56:28,654 DEBUG [Listener at localhost/46205] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-02-11 03:56:28,655 DEBUG [Listener at localhost/46205] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-02-11 03:56:28,658 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/30759f6081f5ddc390558cd75a0299b8/family2/9d98065c542c4813a153663332fc15e4, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/30759f6081f5ddc390558cd75a0299b8/family2/72fa679c42fa42ed913c4140d456aeed, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/30759f6081f5ddc390558cd75a0299b8/family2/0b1c2c1de15a415488c64db1e9c2a924, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/30759f6081f5ddc390558cd75a0299b8/family2/714b21396662486ea76d3db63c3145ea, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/30759f6081f5ddc390558cd75a0299b8/family2/b57223d55d914ecdb19acd5f1f89ef3d] to archive 2023-02-11 03:56:28,659 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-11 03:56:28,660 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=39921] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', ENCRYPTION => 'AES', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', ENCRYPTION_KEY => 'B\x0A\x03AES\x10\x10\x1A\x10\x04DH\xA4>\x17c~\xD3 \xA0\xBBE\xE4\x80j"\x10qhn3cY+H\x9D\x01]\x0E\xCD9\x81\xEE*\x10\xD8\x1C.\x9C\xC4\x8C3\x84et\x0E?\xEE\x8C_\xC42\x03MD5', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}] of the table testMajorCompactionFromAdmin 2023-02-11 03:56:28,661 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/30759f6081f5ddc390558cd75a0299b8/family2/9d98065c542c4813a153663332fc15e4 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testMajorCompactionFromAdmin/30759f6081f5ddc390558cd75a0299b8/family2/9d98065c542c4813a153663332fc15e4 2023-02-11 03:56:28,662 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/30759f6081f5ddc390558cd75a0299b8/family2/72fa679c42fa42ed913c4140d456aeed to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testMajorCompactionFromAdmin/30759f6081f5ddc390558cd75a0299b8/family2/72fa679c42fa42ed913c4140d456aeed 2023-02-11 03:56:28,664 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/30759f6081f5ddc390558cd75a0299b8/family2/0b1c2c1de15a415488c64db1e9c2a924 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testMajorCompactionFromAdmin/30759f6081f5ddc390558cd75a0299b8/family2/0b1c2c1de15a415488c64db1e9c2a924 2023-02-11 03:56:28,665 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/30759f6081f5ddc390558cd75a0299b8/family2/714b21396662486ea76d3db63c3145ea to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testMajorCompactionFromAdmin/30759f6081f5ddc390558cd75a0299b8/family2/714b21396662486ea76d3db63c3145ea 2023-02-11 03:56:28,666 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/30759f6081f5ddc390558cd75a0299b8/family2/b57223d55d914ecdb19acd5f1f89ef3d to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testMajorCompactionFromAdmin/30759f6081f5ddc390558cd75a0299b8/family2/b57223d55d914ecdb19acd5f1f89ef3d 2023-02-11 03:56:28,681 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676087621935] procedure2.ProcedureExecutor(1029): Stored pid=123, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionFromAdmin.mobLock, type=EXCLUSIVE 2023-02-11 03:56:28,682 DEBUG [PEWorker-4] locking.LockProcedure(309): LOCKED pid=123, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionFromAdmin.mobLock, type=EXCLUSIVE 2023-02-11 03:56:28,682 DEBUG [regionserver/jenkins-hbase4:0.Chore.2] throttle.PressureAwareCompactionThroughputController(103): CompactionPressure is 0.0, tune throughput to 50.00 MB/second 2023-02-11 03:56:28,683 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676087621935] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionFromAdmin', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-11 03:56:28,683 INFO [PEWorker-4] procedure2.TimeoutExecutorThread(81): ADDED pid=123, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionFromAdmin.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1676088388683 2023-02-11 03:56:28,683 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676087621935] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-02-11 03:56:29,015 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676087621935] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 3 del files, 12 selected files, and 0 irrelevant files table 'testMajorCompactionFromAdmin' and column 'family1' 2023-02-11 03:56:29,037 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676087621935] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e202302112ba5c8b097424919ae112aa728d151c9_del 2023-02-11 03:56:29,058 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676087621935] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b202302115a6a6fe96a334d89b2b2c7e28a56370f_del 2023-02-11 03:56:29,078 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676087621935] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c202302115664a3bfb3b14bfbbd8fe7c6c8ff7802_del 2023-02-11 03:56:29,079 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676087621935] compactions.PartitionedMobCompactor(350): After merging, there are 3 del files. table='testMajorCompactionFromAdmin' column='family1' 2023-02-11 03:56:29,079 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676087621935] zookeeper.ReadOnlyZKClient(139): Connect 0x560d1fac to 127.0.0.1:53383 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-11 03:56:29,083 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676087621935] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@7312f400, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-11 03:56:29,083 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c4ca4238a0b923820dcc509a6f75849b20230211 for table='testMajorCompactionFromAdmin' and column='family1' 2023-02-11 03:56:29,633 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202302112a4ce90c82cf4c7cb860bef25b6e0c42 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b202302112a4ce90c82cf4c7cb860bef25b6e0c42 2023-02-11 03:56:29,635 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c4ca4238a0b923820dcc509a6f75849b20230211 table='testMajorCompactionFromAdmin' column='family1' 2023-02-11 03:56:29,652 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-11 03:56:29,655 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:38692, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-11 03:56:29,663 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-02-11 03:56:29,665 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:53502, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-02-11 03:56:29,695 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c4ca4238a0b923820dcc509a6f75849b20230211/family1/da12c0ef212f45868141cbaf1b79bb17 first=Optional[1\x00\x00\x00\x01] last=Optional[1\x00\x00\x00\x07] 2023-02-11 03:56:29,696 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionFromAdmin,1,1676087770094.fe3a2eacea8dd8daae28302db14142ed., hostname=jenkins-hbase4.apache.org,37011,1676087618428, seqNum=2 for row 1 with hfile group [{family1,hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c4ca4238a0b923820dcc509a6f75849b20230211/family1/da12c0ef212f45868141cbaf1b79bb17}] 2023-02-11 03:56:29,701 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37011] regionserver.HStore(763): Validating hfile at hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c4ca4238a0b923820dcc509a6f75849b20230211/family1/da12c0ef212f45868141cbaf1b79bb17 for inclusion in fe3a2eacea8dd8daae28302db14142ed/family1 2023-02-11 03:56:29,723 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37011] regionserver.HStore(775): HFile bounds: first=1\x00\x00\x00\x01 last=1\x00\x00\x00\x07 2023-02-11 03:56:29,723 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37011] regionserver.HStore(777): Region bounds: first=1 last=2 2023-02-11 03:56:29,723 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37011] regionserver.HRegion(2445): Flush status journal for fe3a2eacea8dd8daae28302db14142ed: 2023-02-11 03:56:29,724 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37011] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c4ca4238a0b923820dcc509a6f75849b20230211/family1/da12c0ef212f45868141cbaf1b79bb17 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/staging/jenkins__testMajorCompactionFromAdmin__4ia5rd0rnrgiqg13cu43p1egnni9erklpmm9v382erpuqfs8vq6jakn2qc0vhhq2/family1/da12c0ef212f45868141cbaf1b79bb17 2023-02-11 03:56:29,726 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37011] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/staging/jenkins__testMajorCompactionFromAdmin__4ia5rd0rnrgiqg13cu43p1egnni9erklpmm9v382erpuqfs8vq6jakn2qc0vhhq2/family1/da12c0ef212f45868141cbaf1b79bb17 as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/fe3a2eacea8dd8daae28302db14142ed/family1/8581673146754edd8d0dbbe62ccb390d_SeqId_28_ 2023-02-11 03:56:29,726 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37011] regionserver.HStore(849): Loaded HFile hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/staging/jenkins__testMajorCompactionFromAdmin__4ia5rd0rnrgiqg13cu43p1egnni9erklpmm9v382erpuqfs8vq6jakn2qc0vhhq2/family1/da12c0ef212f45868141cbaf1b79bb17 into fe3a2eacea8dd8daae28302db14142ed/family1 as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/fe3a2eacea8dd8daae28302db14142ed/family1/8581673146754edd8d0dbbe62ccb390d_SeqId_28_ - updating store file list. 2023-02-11 03:56:29,753 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37011] regionserver.HStore(882): Loaded HFile hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/fe3a2eacea8dd8daae28302db14142ed/family1/8581673146754edd8d0dbbe62ccb390d_SeqId_28_ into fe3a2eacea8dd8daae28302db14142ed/family1 2023-02-11 03:56:29,754 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37011] regionserver.HStore(855): Successfully loaded hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/staging/jenkins__testMajorCompactionFromAdmin__4ia5rd0rnrgiqg13cu43p1egnni9erklpmm9v382erpuqfs8vq6jakn2qc0vhhq2/family1/da12c0ef212f45868141cbaf1b79bb17 into fe3a2eacea8dd8daae28302db14142ed/family1 (new location: hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/fe3a2eacea8dd8daae28302db14142ed/family1/8581673146754edd8d0dbbe62ccb390d_SeqId_28_) 2023-02-11 03:56:29,754 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37011] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/staging/jenkins__testMajorCompactionFromAdmin__4ia5rd0rnrgiqg13cu43p1egnni9erklpmm9v382erpuqfs8vq6jakn2qc0vhhq2/family1/da12c0ef212f45868141cbaf1b79bb17 2023-02-11 03:56:29,756 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c4ca4238a0b923820dcc509a6f75849b20230211 table='testMajorCompactionFromAdmin' column='family1' 2023-02-11 03:56:29,756 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-11 03:56:29,758 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b202302113342da04ff594f2ab9814585df6a50d0 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b202302113342da04ff594f2ab9814585df6a50d0 2023-02-11 03:56:29,759 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b202302116afa6252614047f28eaaf31c5ede4810 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b202302116afa6252614047f28eaaf31c5ede4810 2023-02-11 03:56:29,760 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b202302116d9da738ee2947068908fb14fc71680c to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b202302116d9da738ee2947068908fb14fc71680c 2023-02-11 03:56:29,760 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b20230211b07eb6f6c0d746748f2f3d5cad956402 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b20230211b07eb6f6c0d746748f2f3d5cad956402 2023-02-11 03:56:29,761 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionFromAdmin 4 expired mob files are deleted 2023-02-11 03:56:29,761 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c4ca4238a0b923820dcc509a6f75849b20230211' 2023-02-11 03:56:29,761 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=c4ca4238a0b923820dcc509a6f75849b20230211 for table='testMajorCompactionFromAdmin' and column='family1' 2023-02-11 03:56:29,761 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c81e728d9d4c2f636f067f89cc14862c20230211 for table='testMajorCompactionFromAdmin' and column='family1' 2023-02-11 03:56:29,922 DEBUG [regionserver/jenkins-hbase4:0.Chore.2] throttle.PressureAwareCompactionThroughputController(103): CompactionPressure is 0.0, tune throughput to 50.00 MB/second 2023-02-11 03:56:30,307 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c2023021139479e9d1c0c4a53b09f23e59ab7256c to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c2023021139479e9d1c0c4a53b09f23e59ab7256c 2023-02-11 03:56:30,308 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20230211 table='testMajorCompactionFromAdmin' column='family1' 2023-02-11 03:56:30,349 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c81e728d9d4c2f636f067f89cc14862c20230211/family1/d9b74790e8074a80901f9363521be913 first=Optional[2\x00\x00\x00\x01] last=Optional[2\x00\x00\x00\x07] 2023-02-11 03:56:30,349 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionFromAdmin,2,1676087770094.30759f6081f5ddc390558cd75a0299b8., hostname=jenkins-hbase4.apache.org,37011,1676087618428, seqNum=2 for row 2 with hfile group [{family1,hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c81e728d9d4c2f636f067f89cc14862c20230211/family1/d9b74790e8074a80901f9363521be913}] 2023-02-11 03:56:30,351 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37011] regionserver.HStore(763): Validating hfile at hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c81e728d9d4c2f636f067f89cc14862c20230211/family1/d9b74790e8074a80901f9363521be913 for inclusion in 30759f6081f5ddc390558cd75a0299b8/family1 2023-02-11 03:56:30,371 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37011] regionserver.HStore(775): HFile bounds: first=2\x00\x00\x00\x01 last=2\x00\x00\x00\x07 2023-02-11 03:56:30,371 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37011] regionserver.HStore(777): Region bounds: first=2 last= 2023-02-11 03:56:30,372 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37011] regionserver.HRegion(2445): Flush status journal for 30759f6081f5ddc390558cd75a0299b8: 2023-02-11 03:56:30,373 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37011] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c81e728d9d4c2f636f067f89cc14862c20230211/family1/d9b74790e8074a80901f9363521be913 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/staging/jenkins__testMajorCompactionFromAdmin__i5ri9ier2gghvitt10la9so06tk5ecu85ueqb3kiasljb2aijaqigf67dnund4ks/family1/d9b74790e8074a80901f9363521be913 2023-02-11 03:56:30,374 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37011] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/staging/jenkins__testMajorCompactionFromAdmin__i5ri9ier2gghvitt10la9so06tk5ecu85ueqb3kiasljb2aijaqigf67dnund4ks/family1/d9b74790e8074a80901f9363521be913 as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/30759f6081f5ddc390558cd75a0299b8/family1/6883c8f8c1be40419e710be5d53248c5_SeqId_28_ 2023-02-11 03:56:30,375 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37011] regionserver.HStore(849): Loaded HFile hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/staging/jenkins__testMajorCompactionFromAdmin__i5ri9ier2gghvitt10la9so06tk5ecu85ueqb3kiasljb2aijaqigf67dnund4ks/family1/d9b74790e8074a80901f9363521be913 into 30759f6081f5ddc390558cd75a0299b8/family1 as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/30759f6081f5ddc390558cd75a0299b8/family1/6883c8f8c1be40419e710be5d53248c5_SeqId_28_ - updating store file list. 2023-02-11 03:56:30,403 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37011] regionserver.HStore(882): Loaded HFile hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/30759f6081f5ddc390558cd75a0299b8/family1/6883c8f8c1be40419e710be5d53248c5_SeqId_28_ into 30759f6081f5ddc390558cd75a0299b8/family1 2023-02-11 03:56:30,403 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37011] regionserver.HStore(855): Successfully loaded hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/staging/jenkins__testMajorCompactionFromAdmin__i5ri9ier2gghvitt10la9so06tk5ecu85ueqb3kiasljb2aijaqigf67dnund4ks/family1/d9b74790e8074a80901f9363521be913 into 30759f6081f5ddc390558cd75a0299b8/family1 (new location: hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/30759f6081f5ddc390558cd75a0299b8/family1/6883c8f8c1be40419e710be5d53248c5_SeqId_28_) 2023-02-11 03:56:30,403 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37011] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/staging/jenkins__testMajorCompactionFromAdmin__i5ri9ier2gghvitt10la9so06tk5ecu85ueqb3kiasljb2aijaqigf67dnund4ks/family1/d9b74790e8074a80901f9363521be913 2023-02-11 03:56:30,405 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20230211 table='testMajorCompactionFromAdmin' column='family1' 2023-02-11 03:56:30,405 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-11 03:56:30,407 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c2023021141c897fe62da403797b0c35b4deb7fda to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c2023021141c897fe62da403797b0c35b4deb7fda 2023-02-11 03:56:30,408 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c202302115b31200b3df7470a889738d32b801104 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c202302115b31200b3df7470a889738d32b801104 2023-02-11 03:56:30,409 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c202302117c820746ccc34c1e9dd444cba8a9501d to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c202302117c820746ccc34c1e9dd444cba8a9501d 2023-02-11 03:56:30,410 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20230211ef0836e42e8e4aed9e6240d5f7dc8b91 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20230211ef0836e42e8e4aed9e6240d5f7dc8b91 2023-02-11 03:56:30,410 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionFromAdmin 4 expired mob files are deleted 2023-02-11 03:56:30,410 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c81e728d9d4c2f636f067f89cc14862c20230211' 2023-02-11 03:56:30,410 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=c81e728d9d4c2f636f067f89cc14862c20230211 for table='testMajorCompactionFromAdmin' and column='family1' 2023-02-11 03:56:30,410 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20230211 for table='testMajorCompactionFromAdmin' and column='family1' 2023-02-11 03:56:31,157 DEBUG [regionserver/jenkins-hbase4:0.Chore.1] throttle.PressureAwareCompactionThroughputController(103): CompactionPressure is 0.0, tune throughput to 50.00 MB/second 2023-02-11 03:56:31,375 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202302114f9e37bff52e4a5c8763948e9c05bc10 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e202302114f9e37bff52e4a5c8763948e9c05bc10 2023-02-11 03:56:31,376 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20230211 table='testMajorCompactionFromAdmin' column='family1' 2023-02-11 03:56:31,415 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/d41d8cd98f00b204e9800998ecf8427e20230211/family1/4424789e612d48b6a12e55121fc3c60a first=Optional[0\x00\x00\x00\x01] last=Optional[0\x00\x00\x00\x07] 2023-02-11 03:56:31,416 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionFromAdmin,,1676087770094.953c5751d1228a55e688b51c4b8bb6fe., hostname=jenkins-hbase4.apache.org,37011,1676087618428, seqNum=2 for row with hfile group [{family1,hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/d41d8cd98f00b204e9800998ecf8427e20230211/family1/4424789e612d48b6a12e55121fc3c60a}] 2023-02-11 03:56:31,418 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37011] regionserver.HStore(763): Validating hfile at hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/d41d8cd98f00b204e9800998ecf8427e20230211/family1/4424789e612d48b6a12e55121fc3c60a for inclusion in 953c5751d1228a55e688b51c4b8bb6fe/family1 2023-02-11 03:56:31,439 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37011] regionserver.HStore(775): HFile bounds: first=0\x00\x00\x00\x01 last=0\x00\x00\x00\x07 2023-02-11 03:56:31,439 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37011] regionserver.HStore(777): Region bounds: first= last=1 2023-02-11 03:56:31,439 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37011] regionserver.HRegion(2445): Flush status journal for 953c5751d1228a55e688b51c4b8bb6fe: 2023-02-11 03:56:31,440 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37011] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/d41d8cd98f00b204e9800998ecf8427e20230211/family1/4424789e612d48b6a12e55121fc3c60a to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/staging/jenkins__testMajorCompactionFromAdmin__oklgtq1kn44hqui1h3uf6fo4tdcdkpu7voi13o2gb8b5s2ervk6edvpm7tmvu6jf/family1/4424789e612d48b6a12e55121fc3c60a 2023-02-11 03:56:31,442 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37011] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/staging/jenkins__testMajorCompactionFromAdmin__oklgtq1kn44hqui1h3uf6fo4tdcdkpu7voi13o2gb8b5s2ervk6edvpm7tmvu6jf/family1/4424789e612d48b6a12e55121fc3c60a as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/953c5751d1228a55e688b51c4b8bb6fe/family1/cf3f8db5418848919522399c4a7b3e48_SeqId_28_ 2023-02-11 03:56:31,442 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37011] regionserver.HStore(849): Loaded HFile hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/staging/jenkins__testMajorCompactionFromAdmin__oklgtq1kn44hqui1h3uf6fo4tdcdkpu7voi13o2gb8b5s2ervk6edvpm7tmvu6jf/family1/4424789e612d48b6a12e55121fc3c60a into 953c5751d1228a55e688b51c4b8bb6fe/family1 as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/953c5751d1228a55e688b51c4b8bb6fe/family1/cf3f8db5418848919522399c4a7b3e48_SeqId_28_ - updating store file list. 2023-02-11 03:56:31,465 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37011] regionserver.HStore(882): Loaded HFile hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/953c5751d1228a55e688b51c4b8bb6fe/family1/cf3f8db5418848919522399c4a7b3e48_SeqId_28_ into 953c5751d1228a55e688b51c4b8bb6fe/family1 2023-02-11 03:56:31,465 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37011] regionserver.HStore(855): Successfully loaded hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/staging/jenkins__testMajorCompactionFromAdmin__oklgtq1kn44hqui1h3uf6fo4tdcdkpu7voi13o2gb8b5s2ervk6edvpm7tmvu6jf/family1/4424789e612d48b6a12e55121fc3c60a into 953c5751d1228a55e688b51c4b8bb6fe/family1 (new location: hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/953c5751d1228a55e688b51c4b8bb6fe/family1/cf3f8db5418848919522399c4a7b3e48_SeqId_28_) 2023-02-11 03:56:31,466 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37011] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/staging/jenkins__testMajorCompactionFromAdmin__oklgtq1kn44hqui1h3uf6fo4tdcdkpu7voi13o2gb8b5s2ervk6edvpm7tmvu6jf/family1/4424789e612d48b6a12e55121fc3c60a 2023-02-11 03:56:31,468 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20230211 table='testMajorCompactionFromAdmin' column='family1' 2023-02-11 03:56:31,468 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-11 03:56:31,471 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e202302117b9b517990204390bbd2619a48d244b9 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e202302117b9b517990204390bbd2619a48d244b9 2023-02-11 03:56:31,472 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e20230211d4c3f669250f4f51bd4f7e95a3d83f42 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e20230211d4c3f669250f4f51bd4f7e95a3d83f42 2023-02-11 03:56:31,472 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e20230211d61b27642992406b913fb12b434f7a3e to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e20230211d61b27642992406b913fb12b434f7a3e 2023-02-11 03:56:31,473 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e20230211ebe88bc7cbf04c0ca3a4f84333645941 to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e20230211ebe88bc7cbf04c0ca3a4f84333645941 2023-02-11 03:56:31,473 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionFromAdmin 4 expired mob files are deleted 2023-02-11 03:56:31,473 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/d41d8cd98f00b204e9800998ecf8427e20230211' 2023-02-11 03:56:31,474 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20230211 for table='testMajorCompactionFromAdmin' and column='family1' 2023-02-11 03:56:31,474 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676087621935] compactions.PartitionedMobCompactor(354): After compaction, there are 3 mob files. table='testMajorCompactionFromAdmin' column='family1' 2023-02-11 03:56:31,474 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676087621935] compactions.PartitionedMobCompactor(364): After a mob compaction with all files selected, archiving the del files for table='testMajorCompactionFromAdmin' and column='family1' 2023-02-11 03:56:31,474 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676087621935] compactions.PartitionedMobCompactor(367): [hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e202302112ba5c8b097424919ae112aa728d151c9_del] 2023-02-11 03:56:31,474 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676087621935] backup.HFileArchiver(360): Archiving compacted files. 2023-02-11 03:56:31,475 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676087621935] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e202302112ba5c8b097424919ae112aa728d151c9_del to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e202302112ba5c8b097424919ae112aa728d151c9_del 2023-02-11 03:56:31,475 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676087621935] mob.MobUtils(508): Table testMajorCompactionFromAdmin 1 expired mob files are deleted 2023-02-11 03:56:31,475 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676087621935] compactions.PartitionedMobCompactor(367): [hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b202302115a6a6fe96a334d89b2b2c7e28a56370f_del] 2023-02-11 03:56:31,476 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676087621935] backup.HFileArchiver(360): Archiving compacted files. 2023-02-11 03:56:31,477 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676087621935] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b202302115a6a6fe96a334d89b2b2c7e28a56370f_del to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b202302115a6a6fe96a334d89b2b2c7e28a56370f_del 2023-02-11 03:56:31,477 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676087621935] mob.MobUtils(508): Table testMajorCompactionFromAdmin 1 expired mob files are deleted 2023-02-11 03:56:31,477 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676087621935] compactions.PartitionedMobCompactor(367): [hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c202302115664a3bfb3b14bfbbd8fe7c6c8ff7802_del] 2023-02-11 03:56:31,477 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676087621935] backup.HFileArchiver(360): Archiving compacted files. 2023-02-11 03:56:31,478 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676087621935] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c202302115664a3bfb3b14bfbbd8fe7c6c8ff7802_del to hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c202302115664a3bfb3b14bfbbd8fe7c6c8ff7802_del 2023-02-11 03:56:31,478 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676087621935] mob.MobUtils(508): Table testMajorCompactionFromAdmin 1 expired mob files are deleted 2023-02-11 03:56:31,478 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676087621935] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionFromAdmin', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-11 03:56:31,479 DEBUG [PEWorker-1] locking.LockProcedure(242): UNLOCKED pid=123, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionFromAdmin.mobLock, type=EXCLUSIVE 2023-02-11 03:56:31,481 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=123, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionFromAdmin.mobLock, type=EXCLUSIVE in 2.7990 sec 2023-02-11 03:56:31,494 DEBUG [Listener at localhost/46205] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-02-11 03:56:31,496 DEBUG [Listener at localhost/46205] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-02-11 03:56:31,498 DEBUG [Listener at localhost/46205] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-02-11 03:56:31,499 DEBUG [Listener at localhost/46205] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-02-11 03:56:31,503 INFO [Listener at localhost/46205] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-11 03:56:31,537 INFO [Listener at localhost/46205] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMajorCompactionFromAdmin Thread=247 (was 256), OpenFileDescriptor=625 (was 648), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=163 (was 129) - SystemLoadAverage LEAK? -, ProcessCount=168 (was 168), AvailableMemoryMB=3164 (was 3151) - AvailableMemoryMB LEAK? - 2023-02-11 03:56:31,537 DEBUG [Listener at localhost/46205] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x6b586bb5 to 127.0.0.1:53383 2023-02-11 03:56:31,538 DEBUG [Listener at localhost/46205] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-11 03:56:31,538 INFO [Listener at localhost/46205] hbase.HBaseTestingUtility(1286): Shutting down minicluster 2023-02-11 03:56:31,538 INFO [Listener at localhost/46205] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-02-11 03:56:31,538 DEBUG [Listener at localhost/46205] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x561bf69b to 127.0.0.1:53383 2023-02-11 03:56:31,538 DEBUG [Listener at localhost/46205] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-11 03:56:31,538 DEBUG [Listener at localhost/46205] util.JVMClusterUtil(237): Shutting down HBase Cluster 2023-02-11 03:56:31,538 DEBUG [Listener at localhost/46205] util.JVMClusterUtil(257): Found active master hash=143749390, stopped=false 2023-02-11 03:56:31,538 INFO [Listener at localhost/46205] master.ServerManager(901): Cluster shutdown requested of master=jenkins-hbase4.apache.org,39921,1676087617326 2023-02-11 03:56:31,540 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-02-11 03:56:31,540 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-02-11 03:56:31,540 INFO [Listener at localhost/46205] procedure2.ProcedureExecutor(629): Stopping 2023-02-11 03:56:31,540 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-11 03:56:31,540 DEBUG [Listener at localhost/46205] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x71412c82 to 127.0.0.1:53383 2023-02-11 03:56:31,540 DEBUG [Listener at localhost/46205] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-11 03:56:31,541 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-11 03:56:31,541 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-11 03:56:31,541 INFO [Listener at localhost/46205] regionserver.HRegionServer(2296): ***** STOPPING region server 'jenkins-hbase4.apache.org,37011,1676087618428' ***** 2023-02-11 03:56:31,541 INFO [Listener at localhost/46205] regionserver.HRegionServer(2310): STOPPED: Shutdown requested 2023-02-11 03:56:31,541 INFO [RS:0;jenkins-hbase4:37011] regionserver.HeapMemoryManager(220): Stopping 2023-02-11 03:56:31,542 INFO [RS:0;jenkins-hbase4:37011] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-02-11 03:56:31,542 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-02-11 03:56:31,542 INFO [RS:0;jenkins-hbase4:37011] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-02-11 03:56:31,542 INFO [RS:0;jenkins-hbase4:37011] regionserver.HRegionServer(3304): Received CLOSE for 6938f3b95bbc15aff8fb4a9d24d9aaf5 2023-02-11 03:56:31,543 INFO [RS:0;jenkins-hbase4:37011] regionserver.HRegionServer(3304): Received CLOSE for 614a4b4b2b88efbc57d9d64c1044323f 2023-02-11 03:56:31,543 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 6938f3b95bbc15aff8fb4a9d24d9aaf5, disabling compactions & flushes 2023-02-11 03:56:31,543 INFO [RS:0;jenkins-hbase4:37011] regionserver.HRegionServer(3304): Received CLOSE for 1e1751a139bc311156cd44006ffe3506 2023-02-11 03:56:31,543 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region ns:testMinorCompaction,,1676087739490.6938f3b95bbc15aff8fb4a9d24d9aaf5. 2023-02-11 03:56:31,543 INFO [RS:0;jenkins-hbase4:37011] regionserver.HRegionServer(3304): Received CLOSE for 3fa91eb317d63eaff7b1021985f3ad00 2023-02-11 03:56:31,543 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on ns:testMinorCompaction,,1676087739490.6938f3b95bbc15aff8fb4a9d24d9aaf5. 2023-02-11 03:56:31,543 INFO [RS:0;jenkins-hbase4:37011] regionserver.HRegionServer(3304): Received CLOSE for 08cc9e8540da3c04f9eb90b5ad80782a 2023-02-11 03:56:31,543 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on ns:testMinorCompaction,,1676087739490.6938f3b95bbc15aff8fb4a9d24d9aaf5. after waiting 0 ms 2023-02-11 03:56:31,543 INFO [RS:0;jenkins-hbase4:37011] regionserver.HRegionServer(3304): Received CLOSE for aa18866afc8385f0b9c9afab2244ee1e 2023-02-11 03:56:31,543 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region ns:testMinorCompaction,,1676087739490.6938f3b95bbc15aff8fb4a9d24d9aaf5. 2023-02-11 03:56:31,543 INFO [RS:0;jenkins-hbase4:37011] regionserver.HRegionServer(3304): Received CLOSE for ebab26be0da503a9bd5db0a862dc884b 2023-02-11 03:56:31,543 INFO [RS:0;jenkins-hbase4:37011] regionserver.HRegionServer(3304): Received CLOSE for 7e59b0be260b97681ec2d05527ed1bde 2023-02-11 03:56:31,543 INFO [RS:0;jenkins-hbase4:37011] regionserver.HRegionServer(3304): Received CLOSE for 9c79e0360165a5fa04cbe6dbc395fafc 2023-02-11 03:56:31,543 INFO [RS:0;jenkins-hbase4:37011] regionserver.HRegionServer(3304): Received CLOSE for 7868535fe478faa7f98625b86eeccc35 2023-02-11 03:56:31,543 INFO [RS:0;jenkins-hbase4:37011] regionserver.HRegionServer(3304): Received CLOSE for 1e125fab80f689690e5f26dc5627861b 2023-02-11 03:56:31,543 INFO [RS:0;jenkins-hbase4:37011] regionserver.HRegionServer(3304): Received CLOSE for cb6565dc6679ffd9e191e6bacbcc214d 2023-02-11 03:56:31,543 INFO [RS:0;jenkins-hbase4:37011] regionserver.HRegionServer(3304): Received CLOSE for 621cd59809a16fd910172a9890d26263 2023-02-11 03:56:31,543 INFO [RS:0;jenkins-hbase4:37011] regionserver.HRegionServer(3304): Received CLOSE for af8a134fc8a9b827f534b0bd6e36ea9a 2023-02-11 03:56:31,543 INFO [RS:0;jenkins-hbase4:37011] regionserver.HRegionServer(3304): Received CLOSE for fe3a2eacea8dd8daae28302db14142ed 2023-02-11 03:56:31,543 INFO [RS:0;jenkins-hbase4:37011] regionserver.HRegionServer(3304): Received CLOSE for 953c5751d1228a55e688b51c4b8bb6fe 2023-02-11 03:56:31,543 INFO [RS:0;jenkins-hbase4:37011] regionserver.HRegionServer(3304): Received CLOSE for 2ed5c51addf313895c503de93f35da1a 2023-02-11 03:56:31,543 INFO [RS:0;jenkins-hbase4:37011] regionserver.HRegionServer(3304): Received CLOSE for 186d87f306418fb25d295d824c1272fe 2023-02-11 03:56:31,543 INFO [RS:0;jenkins-hbase4:37011] regionserver.HRegionServer(3304): Received CLOSE for ea4735f297efbbfcc4cbc0bd3d6e42e0 2023-02-11 03:56:31,543 INFO [RS:0;jenkins-hbase4:37011] regionserver.HRegionServer(3304): Received CLOSE for 30759f6081f5ddc390558cd75a0299b8 2023-02-11 03:56:31,543 INFO [RS:0;jenkins-hbase4:37011] regionserver.HRegionServer(1145): stopping server jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:31,543 DEBUG [RS:0;jenkins-hbase4:37011] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x3c067254 to 127.0.0.1:53383 2023-02-11 03:56:31,543 DEBUG [RS:0;jenkins-hbase4:37011] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-11 03:56:31,544 INFO [RS:0;jenkins-hbase4:37011] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-02-11 03:56:31,544 INFO [RS:0;jenkins-hbase4:37011] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-02-11 03:56:31,544 INFO [RS:0;jenkins-hbase4:37011] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-02-11 03:56:31,544 INFO [RS:0;jenkins-hbase4:37011] regionserver.HRegionServer(3304): Received CLOSE for 1588230740 2023-02-11 03:56:31,545 INFO [RS:0;jenkins-hbase4:37011] regionserver.HRegionServer(1475): Waiting on 21 regions to close 2023-02-11 03:56:31,545 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 1588230740, disabling compactions & flushes 2023-02-11 03:56:31,545 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region hbase:meta,,1.1588230740 2023-02-11 03:56:31,545 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-02-11 03:56:31,545 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-02-11 03:56:31,545 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region hbase:meta,,1.1588230740 2023-02-11 03:56:31,545 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2744): Flushing 1588230740 3/3 column families, dataSize=50.56 KB heapSize=78.23 KB 2023-02-11 03:56:31,545 DEBUG [RS:0;jenkins-hbase4:37011] regionserver.HRegionServer(1505): Waiting on 08cc9e8540da3c04f9eb90b5ad80782a, 1588230740, 186d87f306418fb25d295d824c1272fe, 1e125fab80f689690e5f26dc5627861b, 1e1751a139bc311156cd44006ffe3506, 2ed5c51addf313895c503de93f35da1a, 30759f6081f5ddc390558cd75a0299b8, 3fa91eb317d63eaff7b1021985f3ad00, 614a4b4b2b88efbc57d9d64c1044323f, 621cd59809a16fd910172a9890d26263, 6938f3b95bbc15aff8fb4a9d24d9aaf5, 7868535fe478faa7f98625b86eeccc35, 7e59b0be260b97681ec2d05527ed1bde, 953c5751d1228a55e688b51c4b8bb6fe, 9c79e0360165a5fa04cbe6dbc395fafc, aa18866afc8385f0b9c9afab2244ee1e, af8a134fc8a9b827f534b0bd6e36ea9a, cb6565dc6679ffd9e191e6bacbcc214d, ea4735f297efbbfcc4cbc0bd3d6e42e0, ebab26be0da503a9bd5db0a862dc884b, fe3a2eacea8dd8daae28302db14142ed 2023-02-11 03:56:31,547 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/6938f3b95bbc15aff8fb4a9d24d9aaf5/recovered.edits/30.seqid, newMaxSeqId=30, maxSeqId=1 2023-02-11 03:56:31,549 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed ns:testMinorCompaction,,1676087739490.6938f3b95bbc15aff8fb4a9d24d9aaf5. 2023-02-11 03:56:31,549 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 6938f3b95bbc15aff8fb4a9d24d9aaf5: Waiting for close lock at 1676087791543Running coprocessor pre-close hooks at 1676087791543Disabling compacts and flushes for region at 1676087791543Disabling writes for close at 1676087791543Writing region close event to WAL at 1676087791544 (+1 ms)Running coprocessor post-close hooks at 1676087791548 (+4 ms)Closed at 1676087791549 (+1 ms) 2023-02-11 03:56:31,549 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed ns:testMinorCompaction,,1676087739490.6938f3b95bbc15aff8fb4a9d24d9aaf5. 2023-02-11 03:56:31,549 INFO [regionserver/jenkins-hbase4:0.Chore.2] hbase.ScheduledChore(146): Chore: CompactedHFilesCleaner was stopped 2023-02-11 03:56:31,549 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 614a4b4b2b88efbc57d9d64c1044323f, disabling compactions & flushes 2023-02-11 03:56:31,549 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676087689083.614a4b4b2b88efbc57d9d64c1044323f. 2023-02-11 03:56:31,549 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676087689083.614a4b4b2b88efbc57d9d64c1044323f. 2023-02-11 03:56:31,549 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676087689083.614a4b4b2b88efbc57d9d64c1044323f. after waiting 0 ms 2023-02-11 03:56:31,549 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676087689083.614a4b4b2b88efbc57d9d64c1044323f. 2023-02-11 03:56:31,552 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/614a4b4b2b88efbc57d9d64c1044323f/recovered.edits/58.seqid, newMaxSeqId=58, maxSeqId=55 2023-02-11 03:56:31,554 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676087689083.614a4b4b2b88efbc57d9d64c1044323f. 2023-02-11 03:56:31,554 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 614a4b4b2b88efbc57d9d64c1044323f: Waiting for close lock at 1676087791549Running coprocessor pre-close hooks at 1676087791549Disabling compacts and flushes for region at 1676087791549Disabling writes for close at 1676087791549Writing region close event to WAL at 1676087791550 (+1 ms)Running coprocessor post-close hooks at 1676087791553 (+3 ms)Closed at 1676087791554 (+1 ms) 2023-02-11 03:56:31,554 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676087689083.614a4b4b2b88efbc57d9d64c1044323f. 2023-02-11 03:56:31,554 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 1e1751a139bc311156cd44006ffe3506, disabling compactions & flushes 2023-02-11 03:56:31,554 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testScannerOnBulkLoadRefHFiles,,1676087656456.1e1751a139bc311156cd44006ffe3506. 2023-02-11 03:56:31,554 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testScannerOnBulkLoadRefHFiles,,1676087656456.1e1751a139bc311156cd44006ffe3506. 2023-02-11 03:56:31,554 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testScannerOnBulkLoadRefHFiles,,1676087656456.1e1751a139bc311156cd44006ffe3506. after waiting 0 ms 2023-02-11 03:56:31,554 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testScannerOnBulkLoadRefHFiles,,1676087656456.1e1751a139bc311156cd44006ffe3506. 2023-02-11 03:56:31,556 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=48.28 KB at sequenceid=130 (bloomFilter=false), to=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/hbase/meta/1588230740/.tmp/info/e439e2ef7dba4804b4263bb2b96d8467 2023-02-11 03:56:31,557 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testScannerOnBulkLoadRefHFiles/1e1751a139bc311156cd44006ffe3506/recovered.edits/6.seqid, newMaxSeqId=6, maxSeqId=1 2023-02-11 03:56:31,558 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testScannerOnBulkLoadRefHFiles,,1676087656456.1e1751a139bc311156cd44006ffe3506. 2023-02-11 03:56:31,558 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 1e1751a139bc311156cd44006ffe3506: Waiting for close lock at 1676087791554Running coprocessor pre-close hooks at 1676087791554Disabling compacts and flushes for region at 1676087791554Disabling writes for close at 1676087791554Writing region close event to WAL at 1676087791554Running coprocessor post-close hooks at 1676087791558 (+4 ms)Closed at 1676087791558 2023-02-11 03:56:31,558 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testScannerOnBulkLoadRefHFiles,,1676087656456.1e1751a139bc311156cd44006ffe3506. 2023-02-11 03:56:31,558 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 3fa91eb317d63eaff7b1021985f3ad00, disabling compactions & flushes 2023-02-11 03:56:31,558 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region ns:testMinorCompaction,1,1676087739490.3fa91eb317d63eaff7b1021985f3ad00. 2023-02-11 03:56:31,558 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on ns:testMinorCompaction,1,1676087739490.3fa91eb317d63eaff7b1021985f3ad00. 2023-02-11 03:56:31,558 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on ns:testMinorCompaction,1,1676087739490.3fa91eb317d63eaff7b1021985f3ad00. after waiting 0 ms 2023-02-11 03:56:31,558 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region ns:testMinorCompaction,1,1676087739490.3fa91eb317d63eaff7b1021985f3ad00. 2023-02-11 03:56:31,559 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for e439e2ef7dba4804b4263bb2b96d8467 2023-02-11 03:56:31,562 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/3fa91eb317d63eaff7b1021985f3ad00/recovered.edits/30.seqid, newMaxSeqId=30, maxSeqId=1 2023-02-11 03:56:31,562 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed ns:testMinorCompaction,1,1676087739490.3fa91eb317d63eaff7b1021985f3ad00. 2023-02-11 03:56:31,563 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 3fa91eb317d63eaff7b1021985f3ad00: Waiting for close lock at 1676087791558Running coprocessor pre-close hooks at 1676087791558Disabling compacts and flushes for region at 1676087791558Disabling writes for close at 1676087791558Writing region close event to WAL at 1676087791559 (+1 ms)Running coprocessor post-close hooks at 1676087791562 (+3 ms)Closed at 1676087791562 2023-02-11 03:56:31,563 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed ns:testMinorCompaction,1,1676087739490.3fa91eb317d63eaff7b1021985f3ad00. 2023-02-11 03:56:31,563 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 08cc9e8540da3c04f9eb90b5ad80782a, disabling compactions & flushes 2023-02-11 03:56:31,563 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region ns:testMinorCompaction,2,1676087739490.08cc9e8540da3c04f9eb90b5ad80782a. 2023-02-11 03:56:31,563 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on ns:testMinorCompaction,2,1676087739490.08cc9e8540da3c04f9eb90b5ad80782a. 2023-02-11 03:56:31,563 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on ns:testMinorCompaction,2,1676087739490.08cc9e8540da3c04f9eb90b5ad80782a. after waiting 0 ms 2023-02-11 03:56:31,563 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region ns:testMinorCompaction,2,1676087739490.08cc9e8540da3c04f9eb90b5ad80782a. 2023-02-11 03:56:31,569 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/ns/testMinorCompaction/08cc9e8540da3c04f9eb90b5ad80782a/recovered.edits/30.seqid, newMaxSeqId=30, maxSeqId=1 2023-02-11 03:56:31,570 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed ns:testMinorCompaction,2,1676087739490.08cc9e8540da3c04f9eb90b5ad80782a. 2023-02-11 03:56:31,570 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 08cc9e8540da3c04f9eb90b5ad80782a: Waiting for close lock at 1676087791563Running coprocessor pre-close hooks at 1676087791563Disabling compacts and flushes for region at 1676087791563Disabling writes for close at 1676087791563Writing region close event to WAL at 1676087791563Running coprocessor post-close hooks at 1676087791570 (+7 ms)Closed at 1676087791570 2023-02-11 03:56:31,570 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed ns:testMinorCompaction,2,1676087739490.08cc9e8540da3c04f9eb90b5ad80782a. 2023-02-11 03:56:31,570 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing aa18866afc8385f0b9c9afab2244ee1e, disabling compactions & flushes 2023-02-11 03:56:31,571 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testMajorCompactionWithWeeklyFollowedByMonthly,,1676087622055.aa18866afc8385f0b9c9afab2244ee1e. 2023-02-11 03:56:31,571 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthly,,1676087622055.aa18866afc8385f0b9c9afab2244ee1e. 2023-02-11 03:56:31,571 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthly,,1676087622055.aa18866afc8385f0b9c9afab2244ee1e. after waiting 0 ms 2023-02-11 03:56:31,571 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthly,,1676087622055.aa18866afc8385f0b9c9afab2244ee1e. 2023-02-11 03:56:31,573 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/aa18866afc8385f0b9c9afab2244ee1e/recovered.edits/55.seqid, newMaxSeqId=55, maxSeqId=50 2023-02-11 03:56:31,574 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testMajorCompactionWithWeeklyFollowedByMonthly,,1676087622055.aa18866afc8385f0b9c9afab2244ee1e. 2023-02-11 03:56:31,574 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for aa18866afc8385f0b9c9afab2244ee1e: Waiting for close lock at 1676087791570Running coprocessor pre-close hooks at 1676087791570Disabling compacts and flushes for region at 1676087791570Disabling writes for close at 1676087791571 (+1 ms)Writing region close event to WAL at 1676087791571Running coprocessor post-close hooks at 1676087791574 (+3 ms)Closed at 1676087791574 2023-02-11 03:56:31,574 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testMajorCompactionWithWeeklyFollowedByMonthly,,1676087622055.aa18866afc8385f0b9c9afab2244ee1e. 2023-02-11 03:56:31,574 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing ebab26be0da503a9bd5db0a862dc884b, disabling compactions & flushes 2023-02-11 03:56:31,574 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region hbase:namespace,,1676087620517.ebab26be0da503a9bd5db0a862dc884b. 2023-02-11 03:56:31,575 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on hbase:namespace,,1676087620517.ebab26be0da503a9bd5db0a862dc884b. 2023-02-11 03:56:31,575 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on hbase:namespace,,1676087620517.ebab26be0da503a9bd5db0a862dc884b. after waiting 0 ms 2023-02-11 03:56:31,575 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region hbase:namespace,,1676087620517.ebab26be0da503a9bd5db0a862dc884b. 2023-02-11 03:56:31,575 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2744): Flushing ebab26be0da503a9bd5db0a862dc884b 1/1 column families, dataSize=109 B heapSize=592 B 2023-02-11 03:56:31,602 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-02-11 03:56:31,746 DEBUG [RS:0;jenkins-hbase4:37011] regionserver.HRegionServer(1505): Waiting on 1588230740, 186d87f306418fb25d295d824c1272fe, 1e125fab80f689690e5f26dc5627861b, 2ed5c51addf313895c503de93f35da1a, 30759f6081f5ddc390558cd75a0299b8, 621cd59809a16fd910172a9890d26263, 7868535fe478faa7f98625b86eeccc35, 7e59b0be260b97681ec2d05527ed1bde, 953c5751d1228a55e688b51c4b8bb6fe, 9c79e0360165a5fa04cbe6dbc395fafc, af8a134fc8a9b827f534b0bd6e36ea9a, cb6565dc6679ffd9e191e6bacbcc214d, ea4735f297efbbfcc4cbc0bd3d6e42e0, ebab26be0da503a9bd5db0a862dc884b, fe3a2eacea8dd8daae28302db14142ed 2023-02-11 03:56:31,946 DEBUG [RS:0;jenkins-hbase4:37011] regionserver.HRegionServer(1505): Waiting on 1588230740, 186d87f306418fb25d295d824c1272fe, 1e125fab80f689690e5f26dc5627861b, 2ed5c51addf313895c503de93f35da1a, 30759f6081f5ddc390558cd75a0299b8, 621cd59809a16fd910172a9890d26263, 7868535fe478faa7f98625b86eeccc35, 7e59b0be260b97681ec2d05527ed1bde, 953c5751d1228a55e688b51c4b8bb6fe, 9c79e0360165a5fa04cbe6dbc395fafc, af8a134fc8a9b827f534b0bd6e36ea9a, cb6565dc6679ffd9e191e6bacbcc214d, ea4735f297efbbfcc4cbc0bd3d6e42e0, ebab26be0da503a9bd5db0a862dc884b, fe3a2eacea8dd8daae28302db14142ed 2023-02-11 03:56:31,971 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=102 B at sequenceid=130 (bloomFilter=false), to=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/hbase/meta/1588230740/.tmp/rep_barrier/5b1227d0ea5949f590a57797c8006eeb 2023-02-11 03:56:31,975 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 5b1227d0ea5949f590a57797c8006eeb 2023-02-11 03:56:31,980 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=109 B at sequenceid=7 (bloomFilter=true), to=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/hbase/namespace/ebab26be0da503a9bd5db0a862dc884b/.tmp/info/7c0b342d525c4501a06db22fea7a15f0 2023-02-11 03:56:31,984 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/hbase/namespace/ebab26be0da503a9bd5db0a862dc884b/.tmp/info/7c0b342d525c4501a06db22fea7a15f0 as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/hbase/namespace/ebab26be0da503a9bd5db0a862dc884b/info/7c0b342d525c4501a06db22fea7a15f0 2023-02-11 03:56:31,987 INFO [regionserver/jenkins-hbase4:0.Chore.2] hbase.ScheduledChore(146): Chore: CompactionChecker was stopped 2023-02-11 03:56:31,988 INFO [regionserver/jenkins-hbase4:0.Chore.2] hbase.ScheduledChore(146): Chore: MemstoreFlusherChore was stopped 2023-02-11 03:56:31,988 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/hbase/namespace/ebab26be0da503a9bd5db0a862dc884b/info/7c0b342d525c4501a06db22fea7a15f0, entries=3, sequenceid=7, filesize=4.8 K 2023-02-11 03:56:31,988 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2947): Finished flush of dataSize ~109 B/109, heapSize ~576 B/576, currentSize=0 B/0 for ebab26be0da503a9bd5db0a862dc884b in 413ms, sequenceid=7, compaction requested=false 2023-02-11 03:56:31,992 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/hbase/namespace/ebab26be0da503a9bd5db0a862dc884b/recovered.edits/10.seqid, newMaxSeqId=10, maxSeqId=1 2023-02-11 03:56:31,993 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed hbase:namespace,,1676087620517.ebab26be0da503a9bd5db0a862dc884b. 2023-02-11 03:56:31,993 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for ebab26be0da503a9bd5db0a862dc884b: Waiting for close lock at 1676087791574Running coprocessor pre-close hooks at 1676087791574Disabling compacts and flushes for region at 1676087791574Disabling writes for close at 1676087791575 (+1 ms)Obtaining lock to block concurrent updates at 1676087791575Preparing flush snapshotting stores in ebab26be0da503a9bd5db0a862dc884b at 1676087791575Finished memstore snapshotting hbase:namespace,,1676087620517.ebab26be0da503a9bd5db0a862dc884b., syncing WAL and waiting on mvcc, flushsize=dataSize=109, getHeapSize=576, getOffHeapSize=0, getCellsCount=3 at 1676087791575Flushing stores of hbase:namespace,,1676087620517.ebab26be0da503a9bd5db0a862dc884b. at 1676087791575Flushing ebab26be0da503a9bd5db0a862dc884b/info: creating writer at 1676087791575Flushing ebab26be0da503a9bd5db0a862dc884b/info: appending metadata at 1676087791577 (+2 ms)Flushing ebab26be0da503a9bd5db0a862dc884b/info: closing flushed file at 1676087791577Flushing ebab26be0da503a9bd5db0a862dc884b/info: reopening flushed file at 1676087791985 (+408 ms)Finished flush of dataSize ~109 B/109, heapSize ~576 B/576, currentSize=0 B/0 for ebab26be0da503a9bd5db0a862dc884b in 413ms, sequenceid=7, compaction requested=false at 1676087791988 (+3 ms)Writing region close event to WAL at 1676087791990 (+2 ms)Running coprocessor post-close hooks at 1676087791992 (+2 ms)Closed at 1676087791993 (+1 ms) 2023-02-11 03:56:31,993 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:namespace,,1676087620517.ebab26be0da503a9bd5db0a862dc884b. 2023-02-11 03:56:31,993 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 7e59b0be260b97681ec2d05527ed1bde, disabling compactions & flushes 2023-02-11 03:56:31,993 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676087672040.7e59b0be260b97681ec2d05527ed1bde. 2023-02-11 03:56:31,993 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676087672040.7e59b0be260b97681ec2d05527ed1bde. 2023-02-11 03:56:31,993 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676087672040.7e59b0be260b97681ec2d05527ed1bde. after waiting 0 ms 2023-02-11 03:56:31,993 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676087672040.7e59b0be260b97681ec2d05527ed1bde. 2023-02-11 03:56:31,995 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/7e59b0be260b97681ec2d05527ed1bde/recovered.edits/58.seqid, newMaxSeqId=58, maxSeqId=55 2023-02-11 03:56:31,996 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676087672040.7e59b0be260b97681ec2d05527ed1bde. 2023-02-11 03:56:31,996 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 7e59b0be260b97681ec2d05527ed1bde: Waiting for close lock at 1676087791993Running coprocessor pre-close hooks at 1676087791993Disabling compacts and flushes for region at 1676087791993Disabling writes for close at 1676087791993Writing region close event to WAL at 1676087791993Running coprocessor post-close hooks at 1676087791996 (+3 ms)Closed at 1676087791996 2023-02-11 03:56:31,997 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676087672040.7e59b0be260b97681ec2d05527ed1bde. 2023-02-11 03:56:31,997 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 9c79e0360165a5fa04cbe6dbc395fafc, disabling compactions & flushes 2023-02-11 03:56:31,997 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testMinorCompactionWithMonthlyPolicy,,1676087757896.9c79e0360165a5fa04cbe6dbc395fafc. 2023-02-11 03:56:31,997 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testMinorCompactionWithMonthlyPolicy,,1676087757896.9c79e0360165a5fa04cbe6dbc395fafc. 2023-02-11 03:56:31,997 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testMinorCompactionWithMonthlyPolicy,,1676087757896.9c79e0360165a5fa04cbe6dbc395fafc. after waiting 0 ms 2023-02-11 03:56:31,997 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testMinorCompactionWithMonthlyPolicy,,1676087757896.9c79e0360165a5fa04cbe6dbc395fafc. 2023-02-11 03:56:31,999 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMinorCompactionWithMonthlyPolicy/9c79e0360165a5fa04cbe6dbc395fafc/recovered.edits/52.seqid, newMaxSeqId=52, maxSeqId=1 2023-02-11 03:56:32,000 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testMinorCompactionWithMonthlyPolicy,,1676087757896.9c79e0360165a5fa04cbe6dbc395fafc. 2023-02-11 03:56:32,000 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 9c79e0360165a5fa04cbe6dbc395fafc: Waiting for close lock at 1676087791997Running coprocessor pre-close hooks at 1676087791997Disabling compacts and flushes for region at 1676087791997Disabling writes for close at 1676087791997Writing region close event to WAL at 1676087791997Running coprocessor post-close hooks at 1676087792000 (+3 ms)Closed at 1676087792000 2023-02-11 03:56:32,000 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testMinorCompactionWithMonthlyPolicy,,1676087757896.9c79e0360165a5fa04cbe6dbc395fafc. 2023-02-11 03:56:32,000 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 7868535fe478faa7f98625b86eeccc35, disabling compactions & flushes 2023-02-11 03:56:32,001 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testMajorCompactionWithMonthlyPolicy,,1676087635809.7868535fe478faa7f98625b86eeccc35. 2023-02-11 03:56:32,001 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testMajorCompactionWithMonthlyPolicy,,1676087635809.7868535fe478faa7f98625b86eeccc35. 2023-02-11 03:56:32,001 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testMajorCompactionWithMonthlyPolicy,,1676087635809.7868535fe478faa7f98625b86eeccc35. after waiting 0 ms 2023-02-11 03:56:32,001 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testMajorCompactionWithMonthlyPolicy,,1676087635809.7868535fe478faa7f98625b86eeccc35. 2023-02-11 03:56:32,003 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithMonthlyPolicy/7868535fe478faa7f98625b86eeccc35/recovered.edits/52.seqid, newMaxSeqId=52, maxSeqId=1 2023-02-11 03:56:32,004 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testMajorCompactionWithMonthlyPolicy,,1676087635809.7868535fe478faa7f98625b86eeccc35. 2023-02-11 03:56:32,004 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 7868535fe478faa7f98625b86eeccc35: Waiting for close lock at 1676087792000Running coprocessor pre-close hooks at 1676087792000Disabling compacts and flushes for region at 1676087792000Disabling writes for close at 1676087792001 (+1 ms)Writing region close event to WAL at 1676087792001Running coprocessor post-close hooks at 1676087792003 (+2 ms)Closed at 1676087792004 (+1 ms) 2023-02-11 03:56:32,004 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testMajorCompactionWithMonthlyPolicy,,1676087635809.7868535fe478faa7f98625b86eeccc35. 2023-02-11 03:56:32,004 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 1e125fab80f689690e5f26dc5627861b, disabling compactions & flushes 2023-02-11 03:56:32,004 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testMinorCompactionWithWeeklyPolicy,,1676087646502.1e125fab80f689690e5f26dc5627861b. 2023-02-11 03:56:32,004 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testMinorCompactionWithWeeklyPolicy,,1676087646502.1e125fab80f689690e5f26dc5627861b. 2023-02-11 03:56:32,004 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testMinorCompactionWithWeeklyPolicy,,1676087646502.1e125fab80f689690e5f26dc5627861b. after waiting 0 ms 2023-02-11 03:56:32,004 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testMinorCompactionWithWeeklyPolicy,,1676087646502.1e125fab80f689690e5f26dc5627861b. 2023-02-11 03:56:32,006 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMinorCompactionWithWeeklyPolicy/1e125fab80f689690e5f26dc5627861b/recovered.edits/50.seqid, newMaxSeqId=50, maxSeqId=1 2023-02-11 03:56:32,007 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testMinorCompactionWithWeeklyPolicy,,1676087646502.1e125fab80f689690e5f26dc5627861b. 2023-02-11 03:56:32,007 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 1e125fab80f689690e5f26dc5627861b: Waiting for close lock at 1676087792004Running coprocessor pre-close hooks at 1676087792004Disabling compacts and flushes for region at 1676087792004Disabling writes for close at 1676087792004Writing region close event to WAL at 1676087792005 (+1 ms)Running coprocessor post-close hooks at 1676087792007 (+2 ms)Closed at 1676087792007 2023-02-11 03:56:32,007 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testMinorCompactionWithWeeklyPolicy,,1676087646502.1e125fab80f689690e5f26dc5627861b. 2023-02-11 03:56:32,008 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing cb6565dc6679ffd9e191e6bacbcc214d, disabling compactions & flushes 2023-02-11 03:56:32,008 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testMajorCompactionWithWeeklyPolicy,,1676087661058.cb6565dc6679ffd9e191e6bacbcc214d. 2023-02-11 03:56:32,008 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testMajorCompactionWithWeeklyPolicy,,1676087661058.cb6565dc6679ffd9e191e6bacbcc214d. 2023-02-11 03:56:32,008 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testMajorCompactionWithWeeklyPolicy,,1676087661058.cb6565dc6679ffd9e191e6bacbcc214d. after waiting 0 ms 2023-02-11 03:56:32,008 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testMajorCompactionWithWeeklyPolicy,,1676087661058.cb6565dc6679ffd9e191e6bacbcc214d. 2023-02-11 03:56:32,011 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionWithWeeklyPolicy/cb6565dc6679ffd9e191e6bacbcc214d/recovered.edits/49.seqid, newMaxSeqId=49, maxSeqId=1 2023-02-11 03:56:32,011 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testMajorCompactionWithWeeklyPolicy,,1676087661058.cb6565dc6679ffd9e191e6bacbcc214d. 2023-02-11 03:56:32,011 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for cb6565dc6679ffd9e191e6bacbcc214d: Waiting for close lock at 1676087792008Running coprocessor pre-close hooks at 1676087792008Disabling compacts and flushes for region at 1676087792008Disabling writes for close at 1676087792008Writing region close event to WAL at 1676087792008Running coprocessor post-close hooks at 1676087792011 (+3 ms)Closed at 1676087792011 2023-02-11 03:56:32,012 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testMajorCompactionWithWeeklyPolicy,,1676087661058.cb6565dc6679ffd9e191e6bacbcc214d. 2023-02-11 03:56:32,012 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 621cd59809a16fd910172a9890d26263, disabling compactions & flushes 2023-02-11 03:56:32,012 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testCompactionWithHFileLink,2,1676087711806.621cd59809a16fd910172a9890d26263. 2023-02-11 03:56:32,012 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testCompactionWithHFileLink,2,1676087711806.621cd59809a16fd910172a9890d26263. 2023-02-11 03:56:32,012 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testCompactionWithHFileLink,2,1676087711806.621cd59809a16fd910172a9890d26263. after waiting 0 ms 2023-02-11 03:56:32,012 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testCompactionWithHFileLink,2,1676087711806.621cd59809a16fd910172a9890d26263. 2023-02-11 03:56:32,014 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/621cd59809a16fd910172a9890d26263/recovered.edits/38.seqid, newMaxSeqId=38, maxSeqId=31 2023-02-11 03:56:32,016 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testCompactionWithHFileLink,2,1676087711806.621cd59809a16fd910172a9890d26263. 2023-02-11 03:56:32,016 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 621cd59809a16fd910172a9890d26263: Waiting for close lock at 1676087792012Running coprocessor pre-close hooks at 1676087792012Disabling compacts and flushes for region at 1676087792012Disabling writes for close at 1676087792012Writing region close event to WAL at 1676087792013 (+1 ms)Running coprocessor post-close hooks at 1676087792015 (+2 ms)Closed at 1676087792016 (+1 ms) 2023-02-11 03:56:32,016 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testCompactionWithHFileLink,2,1676087711806.621cd59809a16fd910172a9890d26263. 2023-02-11 03:56:32,016 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing af8a134fc8a9b827f534b0bd6e36ea9a, disabling compactions & flushes 2023-02-11 03:56:32,016 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testScannerOnBulkLoadRefHFiles,2,1676087656456.af8a134fc8a9b827f534b0bd6e36ea9a. 2023-02-11 03:56:32,016 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testScannerOnBulkLoadRefHFiles,2,1676087656456.af8a134fc8a9b827f534b0bd6e36ea9a. 2023-02-11 03:56:32,016 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testScannerOnBulkLoadRefHFiles,2,1676087656456.af8a134fc8a9b827f534b0bd6e36ea9a. after waiting 0 ms 2023-02-11 03:56:32,016 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testScannerOnBulkLoadRefHFiles,2,1676087656456.af8a134fc8a9b827f534b0bd6e36ea9a. 2023-02-11 03:56:32,019 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testScannerOnBulkLoadRefHFiles/af8a134fc8a9b827f534b0bd6e36ea9a/recovered.edits/15.seqid, newMaxSeqId=15, maxSeqId=1 2023-02-11 03:56:32,019 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testScannerOnBulkLoadRefHFiles,2,1676087656456.af8a134fc8a9b827f534b0bd6e36ea9a. 2023-02-11 03:56:32,019 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for af8a134fc8a9b827f534b0bd6e36ea9a: Waiting for close lock at 1676087792016Running coprocessor pre-close hooks at 1676087792016Disabling compacts and flushes for region at 1676087792016Disabling writes for close at 1676087792016Writing region close event to WAL at 1676087792017 (+1 ms)Running coprocessor post-close hooks at 1676087792019 (+2 ms)Closed at 1676087792019 2023-02-11 03:56:32,020 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testScannerOnBulkLoadRefHFiles,2,1676087656456.af8a134fc8a9b827f534b0bd6e36ea9a. 2023-02-11 03:56:32,020 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing fe3a2eacea8dd8daae28302db14142ed, disabling compactions & flushes 2023-02-11 03:56:32,020 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testMajorCompactionFromAdmin,1,1676087770094.fe3a2eacea8dd8daae28302db14142ed. 2023-02-11 03:56:32,020 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testMajorCompactionFromAdmin,1,1676087770094.fe3a2eacea8dd8daae28302db14142ed. 2023-02-11 03:56:32,020 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testMajorCompactionFromAdmin,1,1676087770094.fe3a2eacea8dd8daae28302db14142ed. after waiting 0 ms 2023-02-11 03:56:32,020 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testMajorCompactionFromAdmin,1,1676087770094.fe3a2eacea8dd8daae28302db14142ed. 2023-02-11 03:56:32,023 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/fe3a2eacea8dd8daae28302db14142ed/recovered.edits/30.seqid, newMaxSeqId=30, maxSeqId=1 2023-02-11 03:56:32,023 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testMajorCompactionFromAdmin,1,1676087770094.fe3a2eacea8dd8daae28302db14142ed. 2023-02-11 03:56:32,023 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for fe3a2eacea8dd8daae28302db14142ed: Waiting for close lock at 1676087792020Running coprocessor pre-close hooks at 1676087792020Disabling compacts and flushes for region at 1676087792020Disabling writes for close at 1676087792020Writing region close event to WAL at 1676087792021 (+1 ms)Running coprocessor post-close hooks at 1676087792023 (+2 ms)Closed at 1676087792023 2023-02-11 03:56:32,023 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testMajorCompactionFromAdmin,1,1676087770094.fe3a2eacea8dd8daae28302db14142ed. 2023-02-11 03:56:32,023 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 953c5751d1228a55e688b51c4b8bb6fe, disabling compactions & flushes 2023-02-11 03:56:32,023 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testMajorCompactionFromAdmin,,1676087770094.953c5751d1228a55e688b51c4b8bb6fe. 2023-02-11 03:56:32,023 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testMajorCompactionFromAdmin,,1676087770094.953c5751d1228a55e688b51c4b8bb6fe. 2023-02-11 03:56:32,023 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testMajorCompactionFromAdmin,,1676087770094.953c5751d1228a55e688b51c4b8bb6fe. after waiting 0 ms 2023-02-11 03:56:32,023 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testMajorCompactionFromAdmin,,1676087770094.953c5751d1228a55e688b51c4b8bb6fe. 2023-02-11 03:56:32,026 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/953c5751d1228a55e688b51c4b8bb6fe/recovered.edits/30.seqid, newMaxSeqId=30, maxSeqId=1 2023-02-11 03:56:32,026 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testMajorCompactionFromAdmin,,1676087770094.953c5751d1228a55e688b51c4b8bb6fe. 2023-02-11 03:56:32,026 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 953c5751d1228a55e688b51c4b8bb6fe: Waiting for close lock at 1676087792023Running coprocessor pre-close hooks at 1676087792023Disabling compacts and flushes for region at 1676087792023Disabling writes for close at 1676087792023Writing region close event to WAL at 1676087792024 (+1 ms)Running coprocessor post-close hooks at 1676087792026 (+2 ms)Closed at 1676087792026 2023-02-11 03:56:32,026 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testMajorCompactionFromAdmin,,1676087770094.953c5751d1228a55e688b51c4b8bb6fe. 2023-02-11 03:56:32,026 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 2ed5c51addf313895c503de93f35da1a, disabling compactions & flushes 2023-02-11 03:56:32,026 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testScannerOnBulkLoadRefHFiles,1,1676087656456.2ed5c51addf313895c503de93f35da1a. 2023-02-11 03:56:32,026 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testScannerOnBulkLoadRefHFiles,1,1676087656456.2ed5c51addf313895c503de93f35da1a. 2023-02-11 03:56:32,026 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testScannerOnBulkLoadRefHFiles,1,1676087656456.2ed5c51addf313895c503de93f35da1a. after waiting 0 ms 2023-02-11 03:56:32,026 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testScannerOnBulkLoadRefHFiles,1,1676087656456.2ed5c51addf313895c503de93f35da1a. 2023-02-11 03:56:32,029 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testScannerOnBulkLoadRefHFiles/2ed5c51addf313895c503de93f35da1a/recovered.edits/6.seqid, newMaxSeqId=6, maxSeqId=1 2023-02-11 03:56:32,029 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testScannerOnBulkLoadRefHFiles,1,1676087656456.2ed5c51addf313895c503de93f35da1a. 2023-02-11 03:56:32,029 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 2ed5c51addf313895c503de93f35da1a: Waiting for close lock at 1676087792026Running coprocessor pre-close hooks at 1676087792026Disabling compacts and flushes for region at 1676087792026Disabling writes for close at 1676087792026Writing region close event to WAL at 1676087792027 (+1 ms)Running coprocessor post-close hooks at 1676087792029 (+2 ms)Closed at 1676087792029 2023-02-11 03:56:32,029 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testScannerOnBulkLoadRefHFiles,1,1676087656456.2ed5c51addf313895c503de93f35da1a. 2023-02-11 03:56:32,029 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 186d87f306418fb25d295d824c1272fe, disabling compactions & flushes 2023-02-11 03:56:32,029 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testCompactionWithHFileLink,,1676087711806.186d87f306418fb25d295d824c1272fe. 2023-02-11 03:56:32,029 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testCompactionWithHFileLink,,1676087711806.186d87f306418fb25d295d824c1272fe. 2023-02-11 03:56:32,029 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testCompactionWithHFileLink,,1676087711806.186d87f306418fb25d295d824c1272fe. after waiting 0 ms 2023-02-11 03:56:32,029 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testCompactionWithHFileLink,,1676087711806.186d87f306418fb25d295d824c1272fe. 2023-02-11 03:56:32,033 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/186d87f306418fb25d295d824c1272fe/recovered.edits/38.seqid, newMaxSeqId=38, maxSeqId=31 2023-02-11 03:56:32,034 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testCompactionWithHFileLink,,1676087711806.186d87f306418fb25d295d824c1272fe. 2023-02-11 03:56:32,034 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 186d87f306418fb25d295d824c1272fe: Waiting for close lock at 1676087792029Running coprocessor pre-close hooks at 1676087792029Disabling compacts and flushes for region at 1676087792029Disabling writes for close at 1676087792029Writing region close event to WAL at 1676087792032 (+3 ms)Running coprocessor post-close hooks at 1676087792034 (+2 ms)Closed at 1676087792034 2023-02-11 03:56:32,034 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testCompactionWithHFileLink,,1676087711806.186d87f306418fb25d295d824c1272fe. 2023-02-11 03:56:32,034 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing ea4735f297efbbfcc4cbc0bd3d6e42e0, disabling compactions & flushes 2023-02-11 03:56:32,034 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testCompactionWithHFileLink,1,1676087711806.ea4735f297efbbfcc4cbc0bd3d6e42e0. 2023-02-11 03:56:32,034 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testCompactionWithHFileLink,1,1676087711806.ea4735f297efbbfcc4cbc0bd3d6e42e0. 2023-02-11 03:56:32,034 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testCompactionWithHFileLink,1,1676087711806.ea4735f297efbbfcc4cbc0bd3d6e42e0. after waiting 0 ms 2023-02-11 03:56:32,034 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testCompactionWithHFileLink,1,1676087711806.ea4735f297efbbfcc4cbc0bd3d6e42e0. 2023-02-11 03:56:32,037 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testCompactionWithHFileLink/ea4735f297efbbfcc4cbc0bd3d6e42e0/recovered.edits/38.seqid, newMaxSeqId=38, maxSeqId=31 2023-02-11 03:56:32,037 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testCompactionWithHFileLink,1,1676087711806.ea4735f297efbbfcc4cbc0bd3d6e42e0. 2023-02-11 03:56:32,037 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for ea4735f297efbbfcc4cbc0bd3d6e42e0: Waiting for close lock at 1676087792034Running coprocessor pre-close hooks at 1676087792034Disabling compacts and flushes for region at 1676087792034Disabling writes for close at 1676087792034Writing region close event to WAL at 1676087792035 (+1 ms)Running coprocessor post-close hooks at 1676087792037 (+2 ms)Closed at 1676087792037 2023-02-11 03:56:32,037 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testCompactionWithHFileLink,1,1676087711806.ea4735f297efbbfcc4cbc0bd3d6e42e0. 2023-02-11 03:56:32,037 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 30759f6081f5ddc390558cd75a0299b8, disabling compactions & flushes 2023-02-11 03:56:32,037 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testMajorCompactionFromAdmin,2,1676087770094.30759f6081f5ddc390558cd75a0299b8. 2023-02-11 03:56:32,037 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testMajorCompactionFromAdmin,2,1676087770094.30759f6081f5ddc390558cd75a0299b8. 2023-02-11 03:56:32,037 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testMajorCompactionFromAdmin,2,1676087770094.30759f6081f5ddc390558cd75a0299b8. after waiting 0 ms 2023-02-11 03:56:32,037 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testMajorCompactionFromAdmin,2,1676087770094.30759f6081f5ddc390558cd75a0299b8. 2023-02-11 03:56:32,040 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/default/testMajorCompactionFromAdmin/30759f6081f5ddc390558cd75a0299b8/recovered.edits/30.seqid, newMaxSeqId=30, maxSeqId=1 2023-02-11 03:56:32,040 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testMajorCompactionFromAdmin,2,1676087770094.30759f6081f5ddc390558cd75a0299b8. 2023-02-11 03:56:32,040 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 30759f6081f5ddc390558cd75a0299b8: Waiting for close lock at 1676087792037Running coprocessor pre-close hooks at 1676087792037Disabling compacts and flushes for region at 1676087792037Disabling writes for close at 1676087792037Writing region close event to WAL at 1676087792038 (+1 ms)Running coprocessor post-close hooks at 1676087792040 (+2 ms)Closed at 1676087792040 2023-02-11 03:56:32,040 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testMajorCompactionFromAdmin,2,1676087770094.30759f6081f5ddc390558cd75a0299b8. 2023-02-11 03:56:32,146 DEBUG [RS:0;jenkins-hbase4:37011] regionserver.HRegionServer(1505): Waiting on 1588230740 2023-02-11 03:56:32,346 DEBUG [RS:0;jenkins-hbase4:37011] regionserver.HRegionServer(1505): Waiting on 1588230740 2023-02-11 03:56:32,381 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=2.18 KB at sequenceid=130 (bloomFilter=false), to=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/hbase/meta/1588230740/.tmp/table/04bd8f8f5c0d4816b220625860c5a194 2023-02-11 03:56:32,385 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 04bd8f8f5c0d4816b220625860c5a194 2023-02-11 03:56:32,385 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/hbase/meta/1588230740/.tmp/info/e439e2ef7dba4804b4263bb2b96d8467 as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/hbase/meta/1588230740/info/e439e2ef7dba4804b4263bb2b96d8467 2023-02-11 03:56:32,389 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for e439e2ef7dba4804b4263bb2b96d8467 2023-02-11 03:56:32,389 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/hbase/meta/1588230740/info/e439e2ef7dba4804b4263bb2b96d8467, entries=226, sequenceid=130, filesize=37.9 K 2023-02-11 03:56:32,390 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/hbase/meta/1588230740/.tmp/rep_barrier/5b1227d0ea5949f590a57797c8006eeb as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/hbase/meta/1588230740/rep_barrier/5b1227d0ea5949f590a57797c8006eeb 2023-02-11 03:56:32,393 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 5b1227d0ea5949f590a57797c8006eeb 2023-02-11 03:56:32,393 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/hbase/meta/1588230740/rep_barrier/5b1227d0ea5949f590a57797c8006eeb, entries=1, sequenceid=130, filesize=5.0 K 2023-02-11 03:56:32,394 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/hbase/meta/1588230740/.tmp/table/04bd8f8f5c0d4816b220625860c5a194 as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/hbase/meta/1588230740/table/04bd8f8f5c0d4816b220625860c5a194 2023-02-11 03:56:32,396 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 04bd8f8f5c0d4816b220625860c5a194 2023-02-11 03:56:32,396 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/hbase/meta/1588230740/table/04bd8f8f5c0d4816b220625860c5a194, entries=27, sequenceid=130, filesize=6.6 K 2023-02-11 03:56:32,398 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2947): Finished flush of dataSize ~50.56 KB/51772, heapSize ~78.19 KB/80064, currentSize=0 B/0 for 1588230740 in 853ms, sequenceid=130, compaction requested=false 2023-02-11 03:56:32,401 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/data/hbase/meta/1588230740/recovered.edits/133.seqid, newMaxSeqId=133, maxSeqId=1 2023-02-11 03:56:32,402 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-02-11 03:56:32,402 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed hbase:meta,,1.1588230740 2023-02-11 03:56:32,402 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 1588230740: Waiting for close lock at 1676087791545Running coprocessor pre-close hooks at 1676087791545Disabling compacts and flushes for region at 1676087791545Disabling writes for close at 1676087791545Obtaining lock to block concurrent updates at 1676087791545Preparing flush snapshotting stores in 1588230740 at 1676087791545Finished memstore snapshotting hbase:meta,,1.1588230740, syncing WAL and waiting on mvcc, flushsize=dataSize=51772, getHeapSize=80064, getOffHeapSize=0, getCellsCount=365 at 1676087791545Flushing stores of hbase:meta,,1.1588230740 at 1676087791546 (+1 ms)Flushing 1588230740/info: creating writer at 1676087791546Flushing 1588230740/info: appending metadata at 1676087791550 (+4 ms)Flushing 1588230740/info: closing flushed file at 1676087791550Flushing 1588230740/rep_barrier: creating writer at 1676087791559 (+9 ms)Flushing 1588230740/rep_barrier: appending metadata at 1676087791561 (+2 ms)Flushing 1588230740/rep_barrier: closing flushed file at 1676087791561Flushing 1588230740/table: creating writer at 1676087791975 (+414 ms)Flushing 1588230740/table: appending metadata at 1676087791977 (+2 ms)Flushing 1588230740/table: closing flushed file at 1676087791977Flushing 1588230740/info: reopening flushed file at 1676087792386 (+409 ms)Flushing 1588230740/rep_barrier: reopening flushed file at 1676087792390 (+4 ms)Flushing 1588230740/table: reopening flushed file at 1676087792394 (+4 ms)Finished flush of dataSize ~50.56 KB/51772, heapSize ~78.19 KB/80064, currentSize=0 B/0 for 1588230740 in 853ms, sequenceid=130, compaction requested=false at 1676087792398 (+4 ms)Writing region close event to WAL at 1676087792399 (+1 ms)Running coprocessor post-close hooks at 1676087792402 (+3 ms)Closed at 1676087792402 2023-02-11 03:56:32,402 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:meta,,1.1588230740 2023-02-11 03:56:32,422 DEBUG [regionserver/jenkins-hbase4:0.Chore.2] throttle.PressureAwareCompactionThroughputController(103): CompactionPressure is 0.0, tune throughput to 50.00 MB/second 2023-02-11 03:56:32,547 INFO [RS:0;jenkins-hbase4:37011] regionserver.HRegionServer(1171): stopping server jenkins-hbase4.apache.org,37011,1676087618428; all regions closed. 2023-02-11 03:56:32,554 DEBUG [RS:0;jenkins-hbase4:37011] wal.AbstractFSWAL(932): Moved 1 WAL file(s) to /user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/oldWALs 2023-02-11 03:56:32,554 INFO [RS:0;jenkins-hbase4:37011] wal.AbstractFSWAL(935): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C37011%2C1676087618428.meta:.meta(num 1676087620276) 2023-02-11 03:56:32,558 DEBUG [RS:0;jenkins-hbase4:37011] wal.AbstractFSWAL(932): Moved 1 WAL file(s) to /user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/oldWALs 2023-02-11 03:56:32,558 INFO [RS:0;jenkins-hbase4:37011] wal.AbstractFSWAL(935): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C37011%2C1676087618428:(num 1676087620136) 2023-02-11 03:56:32,558 DEBUG [RS:0;jenkins-hbase4:37011] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-11 03:56:32,558 INFO [RS:0;jenkins-hbase4:37011] regionserver.LeaseManager(133): Closed leases 2023-02-11 03:56:32,559 INFO [RS:0;jenkins-hbase4:37011] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS] on shutdown 2023-02-11 03:56:32,559 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-02-11 03:56:32,560 INFO [RS:0;jenkins-hbase4:37011] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:37011 2023-02-11 03:56:32,567 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,37011,1676087618428 2023-02-11 03:56:32,567 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-11 03:56:32,567 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-11 03:56:32,567 ERROR [Listener at localhost/46205-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@900e63 rejected from java.util.concurrent.ThreadPoolExecutor@4af9aa9[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 533] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-11 03:56:32,570 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,37011,1676087618428] 2023-02-11 03:56:32,570 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,37011,1676087618428; numProcessing=1 2023-02-11 03:56:32,571 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /hbase/draining/jenkins-hbase4.apache.org,37011,1676087618428 already deleted, retry=false 2023-02-11 03:56:32,571 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase4.apache.org,37011,1676087618428 expired; onlineServers=0 2023-02-11 03:56:32,571 INFO [RegionServerTracker-0] regionserver.HRegionServer(2296): ***** STOPPING region server 'jenkins-hbase4.apache.org,39921,1676087617326' ***** 2023-02-11 03:56:32,571 INFO [RegionServerTracker-0] regionserver.HRegionServer(2310): STOPPED: Cluster shutdown set; onlineServer=0 2023-02-11 03:56:32,572 DEBUG [M:0;jenkins-hbase4:39921] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@49ac7b35, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-02-11 03:56:32,572 INFO [M:0;jenkins-hbase4:39921] regionserver.HRegionServer(1145): stopping server jenkins-hbase4.apache.org,39921,1676087617326 2023-02-11 03:56:32,572 INFO [M:0;jenkins-hbase4:39921] regionserver.HRegionServer(1171): stopping server jenkins-hbase4.apache.org,39921,1676087617326; all regions closed. 2023-02-11 03:56:32,572 DEBUG [M:0;jenkins-hbase4:39921] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-11 03:56:32,572 DEBUG [M:0;jenkins-hbase4:39921] cleaner.LogCleaner(198): Cancelling LogCleaner 2023-02-11 03:56:32,572 WARN [OldWALsCleaner-0] cleaner.LogCleaner(186): Interrupted while cleaning old WALs, will try to clean it next round. Exiting. 2023-02-11 03:56:32,572 DEBUG [M:0;jenkins-hbase4:39921] cleaner.HFileCleaner(317): Stopping file delete threads 2023-02-11 03:56:32,572 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1676087619829] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1676087619829,5,FailOnTimeoutGroup] 2023-02-11 03:56:32,572 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1676087619829] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1676087619829,5,FailOnTimeoutGroup] 2023-02-11 03:56:32,572 INFO [M:0;jenkins-hbase4:39921] master.MasterMobCompactionThread(168): Waiting for Mob Compaction Thread to finish... 2023-02-11 03:56:32,573 INFO [M:0;jenkins-hbase4:39921] master.MasterMobCompactionThread(168): Waiting for Region Server Mob Compaction Thread to finish... 2023-02-11 03:56:32,573 INFO [M:0;jenkins-hbase4:39921] hbase.ChoreService(369): Chore service for: master/jenkins-hbase4:0 had [] on shutdown 2023-02-11 03:56:32,573 DEBUG [M:0;jenkins-hbase4:39921] master.HMaster(1502): Stopping service threads 2023-02-11 03:56:32,573 INFO [M:0;jenkins-hbase4:39921] procedure2.RemoteProcedureDispatcher(118): Stopping procedure remote dispatcher 2023-02-11 03:56:32,573 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/master 2023-02-11 03:56:32,573 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-11 03:56:32,573 ERROR [M:0;jenkins-hbase4:39921] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] Thread[HFileArchiver-20,5,PEWorkerGroup] Thread[HFileArchiver-21,5,PEWorkerGroup] Thread[HFileArchiver-22,5,PEWorkerGroup] 2023-02-11 03:56:32,574 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-02-11 03:56:32,574 INFO [M:0;jenkins-hbase4:39921] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-11 03:56:32,574 DEBUG [normalizer-worker-0] normalizer.RegionNormalizerWorker(174): interrupt detected. terminating. 2023-02-11 03:56:32,574 DEBUG [M:0;jenkins-hbase4:39921] zookeeper.ZKUtil(398): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Unable to get data of znode /hbase/master because node does not exist (not an error) 2023-02-11 03:56:32,575 WARN [M:0;jenkins-hbase4:39921] master.ActiveMasterManager(323): Failed get of master address: java.io.IOException: Can't get master address from ZooKeeper; znode data == null 2023-02-11 03:56:32,575 INFO [M:0;jenkins-hbase4:39921] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-11 03:56:32,575 INFO [M:0;jenkins-hbase4:39921] region.MasterRegion(167): Closing local region {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, isAbort=false 2023-02-11 03:56:32,575 DEBUG [M:0;jenkins-hbase4:39921] regionserver.HRegion(1603): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-02-11 03:56:32,575 INFO [M:0;jenkins-hbase4:39921] regionserver.HRegion(1625): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-11 03:56:32,575 DEBUG [M:0;jenkins-hbase4:39921] regionserver.HRegion(1646): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-11 03:56:32,575 DEBUG [M:0;jenkins-hbase4:39921] regionserver.HRegion(1713): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-02-11 03:56:32,575 DEBUG [M:0;jenkins-hbase4:39921] regionserver.HRegion(1723): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-11 03:56:32,575 INFO [M:0;jenkins-hbase4:39921] regionserver.HRegion(2744): Flushing 1595e783b53d99cd5eef43b6debb2682 1/1 column families, dataSize=512.78 KB heapSize=598.48 KB 2023-02-11 03:56:32,670 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-11 03:56:32,670 INFO [RS:0;jenkins-hbase4:37011] regionserver.HRegionServer(1228): Exiting; stopping=jenkins-hbase4.apache.org,37011,1676087618428; zookeeper connection closed. 2023-02-11 03:56:32,670 ERROR [Listener at localhost/46205-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@70c5fc8d rejected from java.util.concurrent.ThreadPoolExecutor@4af9aa9[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 533] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.hadoop.hbase.zookeeper.PendingWatcher.process(PendingWatcher.java:38) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-11 03:56:32,670 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): regionserver:37011-0x10041834dcb0001, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-11 03:56:32,670 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@32da4a1a] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@32da4a1a 2023-02-11 03:56:32,670 ERROR [Listener at localhost/46205-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@8a4a19f rejected from java.util.concurrent.ThreadPoolExecutor@4af9aa9[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 533] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-11 03:56:32,671 INFO [Listener at localhost/46205] util.JVMClusterUtil(335): Shutdown of 1 master(s) and 1 regionserver(s) complete 2023-02-11 03:56:32,982 INFO [M:0;jenkins-hbase4:39921] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=512.78 KB at sequenceid=1006 (bloomFilter=true), to=hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/d79cf1ebe62a44d9a38148c9b154c6e3 2023-02-11 03:56:32,986 INFO [M:0;jenkins-hbase4:39921] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d79cf1ebe62a44d9a38148c9b154c6e3 2023-02-11 03:56:32,986 DEBUG [M:0;jenkins-hbase4:39921] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/d79cf1ebe62a44d9a38148c9b154c6e3 as hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/d79cf1ebe62a44d9a38148c9b154c6e3 2023-02-11 03:56:32,989 INFO [M:0;jenkins-hbase4:39921] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d79cf1ebe62a44d9a38148c9b154c6e3 2023-02-11 03:56:32,989 INFO [M:0;jenkins-hbase4:39921] regionserver.HStore(1080): Added hdfs://localhost:45365/user/jenkins/test-data/615c11aa-528d-0f3e-96e4-2c94e35217ac/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/d79cf1ebe62a44d9a38148c9b154c6e3, entries=123, sequenceid=1006, filesize=33.3 K 2023-02-11 03:56:32,990 INFO [M:0;jenkins-hbase4:39921] regionserver.HRegion(2947): Finished flush of dataSize ~512.78 KB/525088, heapSize ~598.46 KB/612824, currentSize=0 B/0 for 1595e783b53d99cd5eef43b6debb2682 in 415ms, sequenceid=1006, compaction requested=false 2023-02-11 03:56:32,991 INFO [M:0;jenkins-hbase4:39921] regionserver.HRegion(1837): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-11 03:56:32,991 DEBUG [M:0;jenkins-hbase4:39921] regionserver.HRegion(1557): Region close journal for 1595e783b53d99cd5eef43b6debb2682: Waiting for close lock at 1676087792575Disabling compacts and flushes for region at 1676087792575Disabling writes for close at 1676087792575Obtaining lock to block concurrent updates at 1676087792575Preparing flush snapshotting stores in 1595e783b53d99cd5eef43b6debb2682 at 1676087792575Finished memstore snapshotting master:store,,1.1595e783b53d99cd5eef43b6debb2682., syncing WAL and waiting on mvcc, flushsize=dataSize=525088, getHeapSize=612824, getOffHeapSize=0, getCellsCount=1161 at 1676087792576 (+1 ms)Flushing stores of master:store,,1.1595e783b53d99cd5eef43b6debb2682. at 1676087792576Flushing 1595e783b53d99cd5eef43b6debb2682/proc: creating writer at 1676087792576Flushing 1595e783b53d99cd5eef43b6debb2682/proc: appending metadata at 1676087792578 (+2 ms)Flushing 1595e783b53d99cd5eef43b6debb2682/proc: closing flushed file at 1676087792579 (+1 ms)Flushing 1595e783b53d99cd5eef43b6debb2682/proc: reopening flushed file at 1676087792987 (+408 ms)Finished flush of dataSize ~512.78 KB/525088, heapSize ~598.46 KB/612824, currentSize=0 B/0 for 1595e783b53d99cd5eef43b6debb2682 in 415ms, sequenceid=1006, compaction requested=false at 1676087792990 (+3 ms)Writing region close event to WAL at 1676087792991 (+1 ms)Closed at 1676087792991 2023-02-11 03:56:32,993 INFO [master:store-WAL-Roller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-02-11 03:56:32,993 INFO [M:0;jenkins-hbase4:39921] flush.MasterFlushTableProcedureManager(83): stop: server shutting down. 2023-02-11 03:56:32,994 INFO [M:0;jenkins-hbase4:39921] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:39921 2023-02-11 03:56:32,996 DEBUG [M:0;jenkins-hbase4:39921] zookeeper.RecoverableZooKeeper(172): Node /hbase/rs/jenkins-hbase4.apache.org,39921,1676087617326 already deleted, retry=false 2023-02-11 03:56:33,098 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-11 03:56:33,098 INFO [M:0;jenkins-hbase4:39921] regionserver.HRegionServer(1228): Exiting; stopping=jenkins-hbase4.apache.org,39921,1676087617326; zookeeper connection closed. 2023-02-11 03:56:33,099 ERROR [Listener at localhost/46205-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@7b0d7707 rejected from java.util.concurrent.ThreadPoolExecutor@a71b576[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 907] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-11 03:56:33,099 DEBUG [Listener at localhost/46205-EventThread] zookeeper.ZKWatcher(600): master:39921-0x10041834dcb0000, quorum=127.0.0.1:53383, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-11 03:56:33,099 ERROR [Listener at localhost/46205-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@60d9dfa rejected from java.util.concurrent.ThreadPoolExecutor@a71b576[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 907] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.hadoop.hbase.zookeeper.PendingWatcher.process(PendingWatcher.java:38) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-11 03:56:33,101 WARN [Listener at localhost/46205] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-02-11 03:56:33,104 INFO [Listener at localhost/46205] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-02-11 03:56:33,211 WARN [BP-709630106-172.31.14.131-1676087614267 heartbeating to localhost/127.0.0.1:45365] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-02-11 03:56:33,211 WARN [BP-709630106-172.31.14.131-1676087614267 heartbeating to localhost/127.0.0.1:45365] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-709630106-172.31.14.131-1676087614267 (Datanode Uuid f8c64c2d-eaeb-46ff-bb7d-856787967a78) service to localhost/127.0.0.1:45365 2023-02-11 03:56:33,213 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/3712ba8e-7346-8b4d-3d93-a55f0db4c73d/cluster_d15eb7ef-e1c7-b786-c58e-de5925629f9c/dfs/data/data1/current/BP-709630106-172.31.14.131-1676087614267] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-02-11 03:56:33,214 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/3712ba8e-7346-8b4d-3d93-a55f0db4c73d/cluster_d15eb7ef-e1c7-b786-c58e-de5925629f9c/dfs/data/data2/current/BP-709630106-172.31.14.131-1676087614267] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-02-11 03:56:33,240 INFO [Listener at localhost/46205] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-02-11 03:56:33,375 INFO [Listener at localhost/46205] zookeeper.MiniZooKeeperCluster(344): Shutdown MiniZK cluster with all ZK servers 2023-02-11 03:56:33,474 INFO [Listener at localhost/46205] hbase.HBaseTestingUtility(1293): Minicluster is down